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PREFACE 

The  analytical  work  presented  here  is  an  attempt 
to  provide  illumination  in  a field  where  decisions  must  be 
made  concerning  the  expenditures  of  potentially  large 
amounts  of  public  money.  These  decisions  have  been  made 
in  the  past  on  the  basis  of  the  intuition  of  knowledgeable 
and  experienced  people.  It  seems  reasonable  that  these 
intuitive  judgements  may  be  made  with  greater  precision  if 
the  analytical  tools  of  operations  research  can  be  properly 
applied  to  these  questions  of  efficiency  in  allocation  of 
resources. 

It  is  the  author's  hope  that  the  kind  of  careful 
study  presented  here  may  lead  to  a more  efficient,  if  not 
optimum,  disbursement  of  the  public  monies.  It  is  in  this 
spirit  that  this  work  has  been  undertaken. 

Gratitude  is  expressed  to  Captain  Robert  Tripp i 
who  assisted  in  defining  the  study.  Lieutenant  Dwight 
Collins,  who  provided  considerable  insight  into  the 
analytical  methods,  and  finally  to  my  wife,  Florence, 
who  provided  patient  editorial  assistance. 
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ABSTRACT 

This  study  is  an  investigation  of  the  uncertainties 
involved  in  the  prediction  and  measurement  of  Life  Cycle 
Costs.  The  particular  treatment  here  analyzes  Logistic 
Support  Costs,  which  are  a subset  of  the  Life  Cycle  Costs. 
The  Logistics  Supportabil ity  Incentives  which  are  embodied 
in  the  current  General  Dynamics  F-16  contract  are  analyzed 
in  the  light  of  the  stochastic  uncertainties  of  prediction 
and  measurement  of  Logistic  Support  Cost. 

A Monte  Carlo  Simulation  model  is  developed  which 
will  approximate  the  uncertainties  involved  in  obtaining 
a sample  measurement  of  Logistic  Support  Cost  in  a fixed 
length  test. 

The  model  output  is  applied  to  the  problems  of 
determining  appropriate  contractor  rewards  or  penalties, 
investigating  the  feasibility  of  contractor  strategies, 
and  investigating  the  effect  of  ious  test  lengths. 
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Chapter  1 
INTRODUCTION 


Objective 

The  objective  cf  this  research  is  to  investigate  and 
provide  insight  into  certain  Department  of  Defense  reli- 
ability and  maintainability  incentive  contracting  options. 

In  particular,  the  options  to  be  explored  in  detail  are 
the  correction  of  deficiencies  (COD)  and  award  fee  pro- 
visions which  are  embodied  in  the  current  General  Dynamics 
F-16  contract.  The  current  provisions  of  the  F-16  contract 
will  provide  a case  9tudy  framework  within  which  this  en- 
quiry may  be  conducted.  The  products  of  this  study  are 
intended  to  be  useful  not  only  to  the  managers  of  the  F-16 
acquisition,  who  must  formulate  plans  for  administration  of 
their  existing  award  fee  and  correction  of  deficiencies  pro- 
visions; but  also  to  managers  who  must  plan  future  acquisi- 
tions which  will  employ  reliability  and  maintainability 
incentives  in  the  form  of  award  fees  and  corrections  of 
deficiencies  clauses.  The  study  will  attempt  to  quantify 
and  explain  the  considerable  uncertainties  involved  in 
measurement  of  reliability  and  maintainability;  this 
measurement  being  a logically  necessary  prerequisite  to 
any  rational  exercise  of  the  positive  or  negative  incen- 
tive provisions  of  this  form  of  contract. 
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Background 

The  Department  of  Defense  has  entered  a new  era  of 
fiscal  austerity.  In  order  to  maintain  an  effective 
force  within  the  budget  constraints  of  the  future,  the 
Air  Force,  along  with  the  other  departments,  must  conduct 
careful  studies  of  the  total  cost  impact  of  weapons  system 
acquisitions.  One  of  the  tools  for  measurement  of  the 
total  cost  impact  is  the  Life  Cycle  Cost  concept.  Under 
this  concept  the  Air  Force  attempts  to  minimize  the  total 
' Life  Cycle  Cost  of  a weapons  system  while  maintaining  a 
given  effectiveness  level.  In  general,  the  Life  Cycle  Cost 
of  any  system  is  the  sum  of  the  acquisition  cost  and  the 
operating  and  support  cost.  In  this  thesis,  in  order  to 
establish  reasonable  limits  on  the  scope  of  investigation, 
it  has  been  necessary  to  focus  attention  on  a subset  of  the 
operating  and  support  costs.  This  subset,  the  logistic 
support  cost,  will  be  described  in  detail  in  subsequent 
discussions.  It  is  sufficient  here  to  way  that  yie  logis- 
tic support  costs  comprise  a significant  portion  of  the 
overall  operating  and  support;  costs.  The  decision-making 
activities  discussed  in  this  study  will  be  those  which  are 
directly  related  to  logistic  support  cost. 

The  F-16  contract  has  been  structured  with  the  goal 
of  providing  incentive  to  the  contractor  toward  develop- 
ment and  production  of  equipment  which  will  demonstrate 
acceptable  life  cycle  cost  characteristics.  To  be  specific, 
the  F-16  contract  has  been  written  with  both  positive  and 
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negative  incentives  to  the  contractor  associated  with  the 
attainment  of  certain  logistics  supportability  targets. 

The  particular  equipments  to  be  covered  under  the  logis- 
tics supportability  incentives  are  called  first  line  units 
(FLU's).  A FLU  is  defined  to  be  a first,  second,  or  third 
level  of  assembly  described  by  MIL-M-38769-USAF.  A FLU 
is  the  first  levti  or  assembly  below  system  level  that 
would  be  carried  as  a line  item  of  supply  at  base  level. 

A first  line  unit  is  roughly  comparable  to  a line  replace- 
able unit  (LRU) , but  is  a more  precise  definition  which  is 
more  meaningful  and  useful  for  the  purposes  of  measuring 
logistic  support  cost.  In  this  context  a FLU  may  be,  for 
example,  a line  replaceable  unit  in  the  radar  navigation 
system  or  a nose  gear  actuator  in  the  landing  gear  system. 
There  are  two  criteria  for  determining  what  equipi.  ;nts  will 
be  designated  as  FLU's.  The  first  is  that;  within  each 
system,  fault  isolation,  removal,  and  replacement  of  FLU’s 
will  correct  no  less  than  80  per  cent  of  the  failures  in 
that  system.  The  second  criterion  is  that  FLU's  will 
generate  at  least  80  per  cent  of  the  required  maintenance 
manhours  on  the  system.  The  contractor  then  must  rank  his 
components  within  each  system  until  the  sum  of  their  main- 
tenance requirements  will  account  for  both  80  per  cent  of 
the  failures  and  80  per  cent  of  the  direct  maintenance 
manhours.  For  the  F-16  there  are  approximately  280  FLU's. 
These  FLU's  then,  in  theory  at  least,  ought  to  account 
for  at  least  80  per  cent  of  the  maintenance  effort  in  terms 
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of  failures  and  manhours.  These  FLU's  are  a manageable 
and  identifiable  group  of  components  which  should  account 
for  a preponderance  of  the  logistic  support  costs.  To 
provide  even  greater  support  cost  visibility,  a subset  of 
these  280  FLU's  has  been  selected.  The  criterion  for 
selection  of  this  subset,  designated  the  control  FLU's, 
is  that  these  control  FLU's  will  contribute  no  less  than 
50  per  cent  of  the  total  FLU  level  support  costs.  These 
have  been  informally  called  the  "high  burner"  FLU's. 

Different  incentive  arrangements  have  been  made  for 
the  control  and  non-control  FLU's.  For  non-control  FLU's 
an  award  fee  has  been  provided.  This  award  fee  ir.  the 
amount  of  $6,400,000  may  be  paid  fully  or  in  part  to  the 
contractor  at  the  government's  discretion.  If  the  measured 
logistic  support  cost  of  the  non-control  FLU's  is  less  than 
the  government  established  target  logistic  support  cost  of 
the  non-control  FLU's,  then  the  contractor  is  eligible  for 
the  award.  The  measurement  of  the  logistic  support  cost 
will  be  accomplished  during  a verification  test,  wherein 
the  appropriate  input  parameters  for  an  abbreviated  version 
of  the  Air  Force  Logistics  Command  Logistic  Support  Cost 
Model  (AFLC  LSC  model)  will  be  determined.  In  the  case  of 
the  non-control  FLU's,  no  negative  incentive  is  provided. 

The  provisions  covering  the  "high  burner"  control 
FLU '8  are  somewhat  more  complicated.  The  government  has 
three  separate  incentive  alternatives  vis  a vis  the  control 
FLU ' s . 
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The  first  alternative  is  the  Support  Cost  Guarantee 
(SCG) . This  alternative  provides  a positive  incentive  in 
the  form  of  an  award  fee,  and  a negative  incentive  in  the 
form  of  a Correction  of  Deficiencies  (COD)  clause.  The  SCG 
provision  is  the  primary  object  of  investigation  in  this 
study.  Briefly  stated,  the  SCG  alternative  provides  for 
an  award  fee  of  up  to  $2,000,000  in  case  the  measured 
logistic  support  coct  (MLSC)  of  the  control  FLU's  as 
determined  by  the  AFLC  LSC  model  is  less  than  the  govern- 
ment determined  target  logistic  support  cost  (TLSC) . The 
measurement  will  take  place  in  the  above-mentioned  veri- 
fication test.  The  negative  incentive,  correction  of 
deficiencies,  is  invoked  in  case  the  MLSC  is  greater  than 
1.25  times  the  TLSC.  In  this  case  the  contractor  must 
take  action  to  correct  the  deficiency  which  caused  the 
logistic  support  cost  overrun.  The  costs  of  correction 
will  be  shared  with  the  government  on  a 70/30  govern- 
roent/contractor  sharing  ratio  in  accordance  with  the 
provisions  of  the  basic  contract  (1) . The  second  alter- 
native, the  reliability  improvement  warranty  (RIW), 
essentially  provides  for  contractor  maintenance  of  the 
control  FLU's  for  a period  of  48  months  or  300,000  flying 
hours,  whichever  comes  first.  The  third  alternative,  RIW 
with  MTBF  guarantee,  provides  not  only  for  contractor 
maintenance  but  also  for  consignment  (no  charge)  spares 
whenever  the  MTBF  falls  below  the  guaranteed  level. 

It  is  important  to  note  that  eac  of  the  above 
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contracting  alternatives  may  be  applied  on  a FLU  by  FLU 
basis.  The  government  will  review  the  economics  of  each 
control  FLU  and  select  a contracting  option  for  that  FLU. 

If  no  FLU's  are  selected  for  reliability  improvement 
warranties,  then  the  total  possible  award  fee  under  the 
COD  provisions  will  remain  at  $2,000,000.  However,  for 
each  FLU  which  is  chosen  for  an  RIW  option  there  will  be 
a proportionate  reduction  in  the  total  amount  of  the  pos- 
sible award  fee.  This  reduction  will  be  in  the  same 
proportion  as  the  logistic  support  cost  of  the  chosen  FLU 
is  to  the  total  logistic  support  cost.  For  example,  if 
only  one  FLU,  FLU  X,  was  selected  for  a reliability  im- 
provement warranty,  and  if  this  FLU  had  a target  logistic 
support  cost  of  $5.0  million,  then  the  possible  award  fee 
will  be  reduced  by  the  fraction  5.0/TLSC.  Given  a projected 
TLSC  of  $38.4  million,  then  the  total  possible  award  fee 
based  on  the  performance  of  the  remaining  control  FLU's 
would  be: 

$2.0  million  - $2.0  million  x (5.0/38.4) 
or  the  total  possible  award  fee  would  be  $1.74  million. 

If  all  the  control  FLU's  were  selected  for  RIW,  then  of 
course  there  would  be  no  award  fee.* 

There  is  only  one  prerequisite  for  contractor 
eligibility  for  some  or  all  of  the  award  fee.  That 

A similar  reduction  in  the  award  fee  is  made  for  each  FLU 
which  becomes  Government  Furnished  Equipment  (GFS) . 
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prerequisite  is,  as  mentioned  above,  that  the  measured 
logistic  support  cost  be  less  than  the  target  logistic 
support  r>st.  This  single  prerequisite  applies  to  both 
the  control  FLU  award  fee  and  the  non-control  FLU  award 
fee.  Since  there  are  no  further  a priori  conditions,  the 
government  has  complete  discretion  in  determining  the 
amount  of  the  foe,  given  that  the  prerequisite  is  met. 

It  is  one  of  the  stated  objectives  of  this  study  to  pro- 

2 

vide  some  rational  criteria  for  determining  this  amount. 
Figure  1 is  a decision  logic  chart  describing  the  con- 
tracting alternati'/es  and  their  effect  on  the  award  fee. 

Structure  of  the  Problem 

As  has  been  stated,  the  intent  of  the  Air  Force  in 
this  employment  of  the  Life  Cycle  Cost  concept,  is  to 
carefully  measure  the  actual  logistic  support  cost  of  the 
final  product:  or  more  precisely,  to  carefully  measure  a 

visible  and  manageable  subset  of  the  logistic  support  costs 
which  should  comprise  a large  percentage  of  the  total.  To 
this  end  an  abbreviated  logistic  support  cost  model  has 
been  derived  from  the  AFLC  LSC  model . According  to  this 
abbreviated  model,  the  logistic  support  cost  of  a single 
control  FLU  can  be  represented  as  the  sum  of  four  inputs. 
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Figure  1.  Contract  Decision  Logic  Chart 
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These  inputs  are: 

Cl , Initial  and  Replacsmant  Spares 
C2*  On  Equipment  Maintenance  Costs 
C Off  Equipment  Maintenance  Costs 
C^,  Support  Equipment  Coats 
It  will  be  immediately  evident  to  most  observers  that 
these  four  terms  do  not  capture  the  totality  of  logistics 
support  costs  associated  with  each  FLU.  These  four  cost 
terms  have  been  selected  from  the  more  oomprehensive 
AFLC  LSC  model  with  the  purpose  of  providing  a visible, 
measurable  set  of  costs  over  which  the  contractor  should 
be  able  to  exercise  considerable  control.  The  target 
logistic  support  cost  then,  which  will  be  defined  as  the 
sum  of  C^,  C2,  C3  , and  C$  is  not  intended  to  represent  all 
of  the  logistic  support  costs  but  rather  to  stand  as  a 
proxy  for  these  costs.  That  is,  the  target  logistic 
support  cost  is  a representative  standard  against  which 
a contractor's  logistics  performance  may  be  measured.  So, 
even  though  the  model  is  not  suitable  as  a tool  for  measur- 
ing total  costs,  it  is  a useful  device  for  measuring  con- 
tractor performance  in  cost  control. 

The  total  logistic  support  cost  for  the  purposes  of 
this  study  then  is  the  sum  over  all  the  appropriate  PLU's 
of  each  of  these  input  costs. 
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Logistic  Support  Cost  (non-control  flu’s)  *• 
k 

E ( C. j ♦ C*j  + C.j  + C.j  ) 


3-1 
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where  j represents  the  jth  non-control  FLU  and  k is  the 
number  of  non-control  FLU’s. 

Similarly! 

Logistic  Support  Cost  (control  FLU’s)  * 

{ Cli  4 C2i  4 C3i  4 C5i  } 

where  i represents  the  ith  control  FLU  and  n is  the  number 
of  control  FLU’S.3 

Bach  of  the  terms  in  the  above  equations  is  a function 
of  numerous  input  parameters.  The  terms  will  be  described 
in  detail  in  subsequent  chapters.  Among  the  input  param- 
eters there  are  some  which  are  deterministic  and  may  be 
measured  without  error.  An  example  of  one  such  parameter 
is  the  unit  cost  of  initial  spares.  At  the  time  of  the 
verification  test  this  value  will  have  been  negotiated  and 
will  be  a known,  fixed  quantity.  Other  input  parameters, 
for  example,  mean  time  between  failures,  cannot  be  known 
with  certainty.  These  parameters  are  the  means  of 


v ) 


u 


All  further  references  to  logistic  support  cost,  unless 
otherwise  specified*  will  apply  to  control  FLU’s  only. 

As  can  be  clearly  seen  by  the  reader,  the  methodology 
developed  for  the  control  FLU’s  is  readily  generalised 
to  the  other  situations  involving  measurement  or  veri- 
fication of  supportability. 
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probability  distribution*  which  ara  estimated  using  tha 
data  collactad  in  tha  varification  test.  Among  all  tha  in- 
put paramatars  to  tha  LSC  Modal,  there  are  only  13  which  ara 
actually  subject  to  varification  during  tha  varification 
test.  That  is,  wa  ara  interested  in  making  awards,  or 
invoking  the  correction  of  deficiencies  clausa  only  if 
these  "subject  to  verification*  parameters  ara  the  causa 
for  a deviation  from  target  cost.  Clearly,  no  award  or 
penalty  action  should  be  taken  if  a deviation  from  target 
cost  occurs  as  a result  of  an  exogenous  variable  such  as 
inflation  or  a change  in  Air  Force  basing  policy.4 

Among  the  13  parameters  which  are  subject  to  veri- 
fication there  are  three  which  will  be  susceptible  to  the 
greatest  uncertainties  in  measurement.  These  parameters 
aret 

1.  Mean  Time  Between  Failure  (MTBF) 

2.  Maintenance  Man  Hour  Expenditures  (MMH) 

3.  Fraction  of  Failures  Reparable  This  Station  (RTS) 
The  measurements  which  are  taken  during  the  varification 
test  will  actually  provide  estimates  for  MTBF,  MMH,  and 
RTS.  The  reason  that  these  are  estimates  and  not  true 
values  is  that  the  occurrences  of  Time  to  Failure,  Time 

to  Repair,  and  Reparable  This  Station  are  random  variables.5 

4 

Exogenous  in  the  sense  that  it  cannot  be  controlled  by 
the  contractor. 

5A  random  variable  is  a numerical  event  whose  value  will 
vary  in  repeated  samplings  (42:56). 
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Sinoe  estimates  of  MTBF,  MMll,  and  RTS  are  functions  ot  thsse 
random  variable*,  then  they  are  themselves  r.n  d.n  variable*. 
Since  MLSC  is  a function  of  these  estimates  of  MTBF,  MMH , 
and  RTS  it  too  is  a random  variable. 

TVo  hypotheses  have  bee : formed  with  regard  to  these 
three  estimates. 

1.  The  uncertainty  in  these  estimates  is  in  fact  the 
major  contributor  to  the  uncertainty  of  the  m.-usured  logistic 
support  cost. 

2.  The  contributions  to  uncertainty  are,  in  descending 
order  of  magnitude,  estimates  of  MTPF , MMll,  and  RTS.® 

It  is  important  to  note  that  these  hypotheses  make  no 
inference  regarding  the  relative  contributions  of  these 
factors  to  the  total  logistic  support  cost,  but  only  make 
statements  about  how  these  factors  impact  on  the  uncer- 
tainty involved  in  measuring  logistic  support  cost.  It 
does  not  necessarily  follow  that  those  variables  which 
have  the  greatest  impact  on  total  logistic  support  cost 
must  also  have  the  greatest  impact  on  the  uncertainty 
(variance)  of  the  measured  logistic  support  cost. 

In  order  to  develop  a relationship  between  the 
variance  of  the  logistic  support  cost  and  the  input 


The  uncertainty  can  be  thought  of  as  the  amount  of  pos- 
sible variation.  It  will  be  represented  in  this  thesis  by 
the  mathematical  variance  which  is  defined  as  follows:  The 

variance  of  a set  of  observed  measurements,  Y,,...,Y  is 

l n 

the  average  of  the  square  of  the  deviation  of  the  measure- 
ments about  their  mean.  Symbolically:  y(Y)*Z(?-Y  )2/(n-l). 
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parameters  which  are  related  to  randomly  varying  phenomena , 
it  is  necessary  to  hypothesize  some  probability  distribu- 
tion for  each  of  these  random  variables.  Alternatively  an 
attempt  could  be  made  to  fabricate  an  empirical  distri- 
bution; however,  in  this  study,  partially  as  a result  of  the 
paucity  of  historical  data,  the  approach  has  been  to  use 
hypothetical  probability  distributions  which  are  well 
justified  by  both  theory  and  empirical  evidence  where 
available.  When  the  probability  distributions  of  the  input 
parameter  estimates  are  determined,  then  the  problem  is  to 
relate  the  variance  of  the  measurea  logistic  support  cost 
to  the  variance  of  these  estimates.  This  may  be  done 
analytically  if  possible,  or  if  not,  then  it  may  be 
accomplished  by  computer  simulation.  In  this  thesis  the 
relationship  has  been  developed  through  computer  simulation 
due  to  analytical  difficulties  which  will  be  explained  in  a 
subsequent  chapter.  Figure  2 illustrates  the  relationship 
between  input  uncertainty  and  MI^C  uncertainty. 

Realizing  that  the  verification  test  will  be  run  for 
only  a total  of  3500  hours,  it  is  apparent  that  we  will 
be  dealing  with  small  sample  sizes.  For  example,  one  FLU 
which  has  a predicted  MTBF  of  563  hours  would  on  the 
average  experience  about  six  failures  in  3500  hours.  It 
is  not  inconceivable  that  an  item  with  a true  MTBF  of  563 
hours  might  survive  for  3500  hours  with  only  one  or  two 
failures.  Based  on  this  very  small  sample  of  failure*  it 


13 


GQJg/SM/75Qr6 


SOURCE 

UNCERTAINTV 


Time  To 
Failure 


Time  to 
Repair 


LOGISTIC 
SUPPORT 
COST  MODEL 


Fraction 

Reparable 

ThiL 

Station 


Other 

Factors 

If 

Applicable 


ML  SC 

UNCERTAINTY 


Measured 
Logistic 
Support  Cost 


J 

1 

* 


Figure  2.  Uncertainty  Sources 
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is  necessary  to  make  a prediction  of  the  IS  year  life  cycle 
cost  of  the  equipment.  Consider  again  the  FLU  with  the  S63 
hour  true  MTBF.  As  stated  this  FLU  will  have  on  the  average 
six  failures  in  3500  hours.  A 95  per  cent  confidence  in- 
terval for  the  mean  time  between  failures  for  this  FLU  would 
then  be  563±  459;  a very  wide  confidence  interval.^  It  can 
be  seen  then  that  the  3500  hour  test  is  in  essence  a single 
observation  of  the  random  variable,  MLSC,  which  is  based  on 
small  samples  of  input  random  variables.  It  would  not  be 
surprising  then  to  find  that  the  MLSC  has  a relatively  large 
variance. 

Once  the  variance  and  the  type  of  probability  distri- 
bution for  the  MLSC  have  been  determined,  then  it  is  pos- 
sible to  make  statistical  inferences  based  on  the  results 
of  a single  3500  hour  test.  For  example,  as  will  be  shown, 
it  is  possible,  given  a single  sample  observation  of  MLSC, 
to  say  with  what  level  of  confidence  that  particular  obser- 
vation implies  either  a real  target  cost  underrun  or  a real 
logistics  support  cost  in  excess  of  1.25  times  target  cost. 
That  is  to  say,  we  can  derive  information  which  will  be  use- 
ful in  determining  the  probability  of  a correct  decision. 


Using  the  approximation:  o__ . . . 3 o//n  where  n ■ number  ol 

SfiUn  failures 

of  mean 
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Exponential 
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By  varying  the  length  of  the  test  above  and  below 
3500  hour3  in  the  simulation  model  some  additional  infor- 
mation can  be  derived.  For  example,  we  can  determine  how 
much  an  increase  of  1000  hours  of  test  time  will  increase 
the  probability  of  making  a correct  uocision. 

Using  these  concepts,  a series  of  decision  curves  can 
be  constructed.  These  curves  will  show  for  a given  test 
length  how  much  confidence  can  be  placed  in  any  single 
observation  of  MLSC. 

In  view  of  the  embarrassment  to  the  Air  Force  which 
would  result  from  the  decision  to  present  an  award  fee  to  a 
contractor  whose  product  subsequently  showed  excessive 
support  costs,  it  is  clear  that  this  decision  ought  to 
'Tbe  made  with  the  benefit  of  some  statistical  analysis. 
Conversely,  the  Air  Force  certainly  would  not  want  to 
invoke  a correction  of  deficiencies  without  a reasonable 
confidence  that  a support  cost  overrun  has  occurred. 

Indeed,  since  the  Air  Force  must  pay  70  per  cent  of  the 
costs  of  correction  of  deficiencies,  an  erroneous  invo- 
cation of  COD  would  unfairly  penalize  not  only  the  con- 
tractor but  also  the  government.  Since  the  COD  pro- 
visions are  thoroughly  specified  in  the  F-16  contract, 
this  study  can  provide  statistical  knowledge  of  the 
likelihood  that  a given  decision  to  invoke  COD  or  make  an 
award  was  correct.  Further,  as  will  be  subsequently  shown, 
the  methods  developed  here  can  provide  substantial  insight 
to  those  who  must  develop  the  correction  of  deficiencies 
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provisions  for  future  acquisitions. 

While  conducting  an  investigation  of  the  uncertainties 
involved  in  measurement  of  logistic  support  cost,  some 
useful  by-products  are  obtained.  Using  the  mathematical 
model  developed  in  this  study  it  is  possible  to  conduct 
numerous  sensitivity  analyses.  Some  examples  are: 
sensitivity  of  the  variance  of  MLSC  to  the  number  of  FLU'S 
in  the  test  and  sensitivity  of  LSC  to  stock  level  policy. 
These  two  analyses  are  shown  in  Appendices  A and  C re- 
spectively. 

Since  it  appears  that  the  Life  Cycle  Cost  concept  is 
here  to  stay,  and  inasmuch  as  the  methodology  to  be  de- 
scribed here  may  be  readily  generalized,  it  is  apparent 
that  these  techniques  ought  to  be  of  significant  value  to 
the  Department  of  Defense. 

Nature  and  Sources  of  Data 

The  primary  source  of  data  for  the  F-16  implementation 
of  this  study  is  the  F-16  acquisition  contract.  Hence,  the 
source  of  most  of  the  data  is  the  contractor.  A brief 
discussion  of  the  strengths  and  weaknesses  of  this  data 
follows. 

Consider  first  the  contractor  furnished  MTBF  pre- 
dictions for  the  control  FLU's.  Most  of  these  predictions 
are  based  at  least  in  part  on  similarity  to  some  existing 
Air  Force  equipment  on  either  the  A7D  or  the  F-lll.  Field 
MTBF  data  on  this  baseline  equipment  has  then  been  modified 
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by  a complexity  factor.  For  example,  if  a particular  piece 
Of  equipment  he*  30  per  cent  fewer  components  than  the 
baeeldne  FLU,  the  complexity  factor  might  be  something  like 
d.4.  That  ie,  the  F-16  FLU  would  be  credited  with  an  MTBF 
of  about  1.4  times  the  base’jne  FLU.  This  figure  would  be 
further  modified  by  the  application  of  a usage  and  environ- 
mental factor,  which  attempts  to  reflect  the  impact  of 
anticipated  environmental  stresses  on  the  MTBF. 

Maintenance  manhour  figures  used  were  based  on  similar 
aircraft  data,  modified  by  complexity  and  size  relationship 
'factors  established  by  the  General  Dynamics  support  require- 
ments division.  The  G.D.  Base  shop  simulator  model  was 
used  in  this  analysis  (1) . 

An  assessment  of  the  validity  of  these  predictions  is 
a highly  subjective  exercise.  Experts  in  the  reliability 
and  maintainability  field  have  stated  the  following  with 
regard  to  predictions  based  on  historical  data. 

If  the  same  organization,  employing  the  same  per- 
sonnel has  demonstrated  an  effective  and  extensive 
quality  control  organization. .. it  is  probably  safe  to 
assume  that  the  reliability  inherent  in  the  new 
system  will  not  be  degraded  as  the  design  is  trans- 
lated from  drawing  to  hardware  any  more  than  was  the 
case  in  earlier  programs  [35]. 

If  we  accept  this  assertion,  and  if  the  stated  preconditions 
are  met,  it  would  seem  reasonable  to  compare  the  field  per- 
formance of  F-lll  equipment  to  the  predictions  made  by 
General  Dynamics.  A review  of  the  F-lll  category  II  test 
report  indicates  that  for  the  eight  avionics  equipments  for 
which  MTBF  predictions  were  provided,  the  measured  MTBF 
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was  on  tha  average  equal  to  one-third  of  the  prediction  (51). 
Considering  this,  and  the  history  of  MTBF  predictions,  where 
according  to  experts  in  the  field,  it  is  not  uncommon  to  see 
an  order  of  magnitude  reduction  in  MTBF  from  laboratory  to 
field  conditions,  it  seems  prudent  to  investigate  the  be- 
havior of  the  LSC  model  with  MTBF ’ s degraded  from  those 
predicted  by  the  contractor.  This  has  been  accomplished 
as  a part  of  the  analysis  (52:27). 

Projecting  realistic  maintainability  estimates  for 
tactical  fighter  aircraft  during  conceptual  and  develop- 
ment design  phases  is  a continuing  problem  for  the  Depart- 
ment  of  Defense.  Figure  3 gives  a comparison  of  predicted 
to  actual  maintenance  manhours  per  flight  hour  (52:18). 


It  is  clear  that  overly  optimistic  estimates  are  prevalent 
in  maintainability  as  well  as  in  reliability.  On  the  aver- 
age for  the  samples  above,  contractor  estimates  have  been 
about  70  per  cent  of  actual  required.  The  General  Dynamics 


19 


GOH/8M/75D-6 


•stimats  for  tha  F-lll  was  approximately  65  per  cent  of  the 
actual  field  NMH/FH  requirements. 

Mere  are  clearly  uncertainties  regarding  the  Accuracy 
of  predicted  reliability  and  maintainability  levels.  The 
purpose  of  the  verification  test  io  to  determine  whether 
the  contractor  has  come  acceptably  close  to  the  predicted 
values . 

Summary  of  the  Methodology 

The  first,  and  in  a sense,  the  most  important  step  in 
this  analysis  is  the  determination  of  the  important  random 
variables  in  the  cost  equations.  That  is,  among  the  input 
parameters  which  are  subject  to  verification,  which  are 
significant  in  terms  of  their  impact  on  MJ..SC  uncertainty? 
And,  among  those  parameters,  which  can  bo  considered 
deterministic  constants  as  opposed  to  random  variables? 

So,  to  be  of  interest  in  this  methodology,  the  parameter 
must  pass  two  tests. 

1.  Is  the  parameter  subject  to  verification? 

2.  Is  the  parameter  the  mean  of  a probability  dis- 
tribution of  a random  variable? 

The  LSC  input  parameters  which  are  subject  to  verifi- 
cation are  listed  below. 


1. 

N: 

number  of 

FLUs 

2. 

QPA: 

Quantity 

of  like  FLUs 

within  the  system. 

3. 

K: 

number  of 

lino  items  of 

peculiar  support 

equipment  (AGE)  for  ith  FLU. 
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4.  MTBFt  Mean  time  between  failures  of  the  FLU  in 
operating  hours,  in  the  operational  environment. 

5.  RIP:  Fraction  of  FLU  failures  reparable  in  place. 

6.  RTS:  Fraction  of  removals  reparable  at  base  level. 

7 . NRTS : 1-RTS . 8 

8.  COND:  Fraction  of  removals  resulting  in  condem- 

nation at  base  level. 

9.  All  Maintenance  manhour  data:  (IMH,PAMH,BMH,DMH) . 

10.  DOWN:  Percentage  of  down  time  for  the  jth  piece 

of  peculiar  age. 

11.  UC:  Expected  unit  cost  of  the  FLU  at  initial 

provisioning. 

12:  CAB:  Cost  per  unit  of  support  equipment  at  base 

level. 

13.  CAD:  Cost  per  unit  of  support  equipment  at  depot. 

It  is  immediately  apparent  that  N,  QPA,  and  K are  known 
constants.  It  has  been  previously  stated  that  the  value  of 
UC  will  be  a known  quantity  at  the  time  of  the  verification 
test.  Contractor  data  indicates  that  the  value  of  K is 
zero,  i.e.f  there  are  no  items  of  peculiar  AGE  for  any 
control  FLU.  Since  K*0,  we  can  eliminate  DOWN,  CAB,  and 
CAD,  all  of  which  apply  to  peculiar  AGE. 


8In  general  it  would  not  be  correct  to  say  that  NRTS»1-RTS 
because;  of  that  fraction  of  failures  which  is  not  reparable 
this  station,  some  portion  may  be  condemned  at  base  level. 

So  that,  in  general,  NRTS-1-RTS-C0ND.  In  this  application 
COND-O  for  all  control  FLUr. 


21  1 


GOR/SM/75D-6 

This  leaves  as  candidates  Cor  inputs  MTHF , RIP , ffT6, 
COND,  PAMH>  RMH,  BMH , and  DMH . 

COND  is  sero  for  all  control  flu a so  it  is  eliminated. 
RIP  is  zero  for  all  but  one  control  Fiu,  the  flight  control 
computer,  for  which  the  value  is  .Ox.  Since  this  input 
would  have  an  insignificant  impact  on  the  overall  uncer- 
tainty, it  is  treated  as  a deterministic  variable.  Since 
the  value  of  RIP  is  near  zero,  the  eggregate  value  of  lid 
(in  place  repair  manhours)  will  also  have  an  insignificant 
impact  on  the  overall  uncertainty  and  thus  it  can  be 
approximated  by  a deterministic  variable 

Those  parameters  which  will  be  used  in  connection  with 
random  variables  then  are:  MTBF,  RTS,  PAKH,  BMH,  PJ4H,  and 

DMH.9 

Having  identified  these  parameters,  the  next  step  in 
this,  or  any  similar  analysis,  is  to  determine  the  prob- 
ability distribution  functions  (PDF's)  of  the  random 
variables  to  which  these  parameters  are  related.  There 
are  essentially  two  approaches  to  this  task;  the  theoretical 
approach,  and  the  empirical  approach. 

In  the  empirical  approach,  the  preferred  procedure 
would  be  to  collect  reliability  and  maintainability  data  on 
each  of  the  FLUs  and  construct  empirical  distribution 

5 

NRTS  must  be  treated  in  the  same  manner  but  it  will  not 
be  explicitly  represented  since  NRTS*1-RTS. 
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functions  using  this  data.  It  is  very  likely  though  that 
this  information  may  be  unavailable  or  nonexistent,  partic- 
ularly for  new  systems.  The  data  collected  by  the  Air  Force 
manual  66-1  system  is  aggregate  data  which  is  useless  for 
purposes  of  constructing  histograms  of  time  to  failure  or 
time  to  repair.  An  alternative  approach  which  makes  use  of 
some  historical  data  is  the  use  of  category  II  test  and 
evaluation  information  from  similar  aircraft  systems. 

These  test  and  evaluation  reports,  published  by  the  Air 
Force  Flight  Test  Center,  contain  empirical  probability 
distributions  for  time  to  failure  and  time  to  repair  on  the 
important  subsystems  of  the  aircraft.  From  this  information 
some  reasonable  inferences  can  be  made  about  new  systems  if 
the  systems  are  similar  (49;  50;  51). 

In  a purely  theoretical  approach  it  would  be  appro- 
priate to  simply  hypothesize  a distribution  which,  according 
to  theory,  should  represent  the  process  in  question.  Con- 
siderable discussion  of  this  question  is  available  in  the 
literature  of  reliability  and  maintainability. 

This  analysis  has  drawn  on  information  from  both  of 
these  sources.  In  the  initial  stages  of  the  analysis,  PDF's 
were  chosen  for  time  to  failure  and  time  to  repair  based  on 
theory.  As  historical  information  became  available  in  the 
form  of  test  reports  for  the  A7D  and  the  F-lll,  it  largely 
confirmed  the  theoretical  distributions.  When  there  was 
reasonable  doubt  as  to  the  correct  distribution,  sensitivity 
analysis  was  conducted  to  determine  what  would  be  the  impact 
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of  an  error  in  choice  of  distributions.  The  same  general 
approach  has  been  used  to  determine  appropriate  distribu- 
tions for  both  NTBP's  and  maintenance  manhours.  There  is, 
of  course,  a myriad  of  diverse  tasks  associated  with  each 
PLU  failure,  but  it  is  not  tne  purpose  of  this  undertaking 
to  study  each  of  these  maintenance  activities  in  dstsil. 
Rather,  the  approach  ha6  been  to  assume  that  all  mainte- 
nance activities  can  be  fairly  represented  by  n single  PDF. 

A final  assumption  was  made  regarding  time  to  failure 
and  time  to  repair.  It  has  been  assumed  here  that  the  mean 
of  each  distribution  will  be  constant  over  the  period  of 
the  test.  This  means  no  allowance  will  be  made  tor  MTBF 
growth  or  a personnel  learning  curve  during  the  six  month 
period  of  the  test.  Information  from  the  inertial  navigation 
contractor  is  somewhat  at  variance  with  this  assumption  ( 57 i . 
This  contractor  has  planned  for  considerable  MTfiF  growth 
during  the  initial  operational  period.  The  planned  growth 
rate  if  assumed  to  be  linear  would  indicate  a change  of  17 
per  cent  in  the  INS  MTBF  during  the  six  months  of  the  veri- 
fication test.  The  assumption  here  is  that  this  discovery 
does  not  do  great  violence  to  the  findings  of  this  report. 

In  order  to  formulate  a probability  distribution  for 
RTS  it  is  only  necessary  to  visualize  this  process  as  a 
Bernoulli  trial  such  as  the  toss  of  a biased  com.  This  is 
essentially  the  process  which  occurs  with  each  FLU  failure. 
With  this  in  mind  there  is  no  alternative  other  than  the 
binomial  distribution. 
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With  all  of  tha  abova  decisions  mads , tha  naxt  step  in 
tha  analyaia  ia  to  estimate  tha  axpactad  valua  and  varianca 
(B  (MLSC)  and  V (MLSC) ) of  tha  measured  logiatic  aupport  coat 
•a  a function  of  the  input  parameter c MTBF,  mmh,  and  RTS. 

That  iat 


B (MLSC  - f (MTBF , 

MMH, 

RTS) 

AND 

V (MLSC  - F (MTBF, 

MMH, 

RTS) 

Tha  difficulties  associated  with  an  analytical  approach  to 
the  above  problem  will  depend  on  the  underlying  probability 
distributions.  In  this  particular  application  of  the 
SMthodology  the  stumbling  block  to  analytical  expression 
is  in  the  stock  level  equations  which  will  be  covered  in 
detail  in  Chapter  4.  For  this  application,  Monte  Carlo 
Simulation  has  been  used  in  this  part  of  the  analysis. 

Although  the  simulation  model  will  be  covered  in 
detail  in  Chapter  4,  a brief  summary  description  is  pre- 
sented here.  The  simulation  model  essentially  duplicates 
the  transactions  and  conditions  of  the  3500  hour  verifi- 
cation test.  The  model  first  generates  random  failures 
for  each  control  FLU  over  a 3500  hour  period,  based  on 
either  an  exponential  or  a Weibull  distribution  of  failures 
as  selected  by  user.  For  each  failure  of  the  ith  FLU,  the 
model  then  samples  from  the  binomial  distribution  to  deter- 
mine whether  the  failure  will  be  reparable  at  this  station 
or  not  reparable  this  station.  For  each  failure,  the 
simulation  samples  from  a manhour  generator  to  determine 
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preparation  and  access  manhours  (PAMH) , ana  removal  man- 
hours (RMh) . This  sample  is  taken  from  cither  e lognormal 
or  a Weibull  distribution  as  chosen  by  the  user.  No 
stochastic  sample  is  cakon  for  in  place  manhours  umu  as 
this  is  treated  as  a deterministic  variable.  Next,  for 
those  failures  which  have  been  determined  to  u reparable 
at  base  level,  the  model  generates  a sample  for  base  man- 
hours (BMri)  from  the  appropriate  manhour  generator.  For 
those  failures  which  were  not  reparable  at  base,  the  model 
generates  a sample  for  manhours  to  repair  at  aepot  (UMH). 

At  the  conclusion  of  each  3500  hour  test,  the  model 
calculates  an  estimate  for  Ml  BF , PAMH,  RMh,  jMh,  uMh , and 
RTS.  Given  these  mean  values  the  model  then  calculates  an 
appropriate  stock  level  for  initial  spares  for  the-  ith  FLU 
based  on  a Poisson  demand  rate  and  a government  established 
expected  backorder  value  (CBO) . 

Finally,  ell  of  this  information  is  inpul  to  the  AFLC 
abbreviated  LSC  model  which  outputs  one  son  pie  point  of  MLSC- 

Since  it  is  certain  that  not  all  readers  will  agree  as 
to  the  propriety  of  the  stochastic  inputs  to  the  simulation, 
it  is  necessary  to  conduct  an  analysis  to  determine  the 
sensitivity  of  the  model  to  the  form  of  the  inputs.  All 
possible  combinations  of  the  inputs  where  each  is  allowed 
to  be  deterministic  or  stochastic  produced  eight  data  points. 
If  the  verification  test  length  is  treated  as  an  input,  then 
there  are  16  data  points.  Standard  analysis  of  variance 
techniques  have  been  employed  in  Chapter  4 to  examine  the 
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affects  end  interactions  of  each  input  variable.  In  addi- 
tion, a separate  analysis  was  conducted  to  determine  the 
sensitivity  of  V(MLSC)  to  the  type  of  PDF  employed  for 
failure  and  manhour  generators.  These  analyses  will 
illuminate  not  only  the  sensitivity  of  the  model  to  the 
assumptions,  but  also  the  contribution  of  each  of  the  input 
random  variables  to  the  overall  uncertainty  of  the  output. 
This  information  is  useful  in  determining  the  relative 
importance  of  careful  measurement  for  each  of  the  input 
random  variables. 

Determination  of  the  form  of  the  MLSC  distribution  is 
accomplished  by  goodness  of  fit  tests.  The  Kolmogorov- 
Smirnov  test  has  been  employed  and  the  results  are  shown 
in  Chapter  4. 

After  the  form  of  the  distribution  is  determined,  then 
statistical  inferences  can  be  made  based  on  a single  obser- 
vation of  MLSC  and  a "decision  curve"  constructed. 

The  final  step  in  the  implementation  of  the  method- 
ology is  the  construction  of  an  award  fee  curve.  This  curve 
is  constructed  with  the  benefit  of  Knowledge  derived  from 
the  decision  curves  and  with  the  provision  for  subjective 
inputs  from  performance  evaluation. 

As  a result  of  this  investigation,  it  has  been  possible 
to: 

1.  Describe  the  major  factors  in  the  uncertainty  of 
life  cycle  cost  estimates  and  their  relative  importance 
as  embodied  ir  the  F-16  contract. 
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2.  Describe  the  probability  distribution  function  of 
the  measured  logistics  support  cost  for  various  input  dis- 
tributions and  as6umptio:.s . 

3.  Determine  the  effect  of  changes  in  verification 
test  length  on  the  uncertainty  of  the  measured  logistic 
support  cost. 

4.  Provide  planning  factors  for  managers  who  will 
formulate  future  reliability  incentive  contracts  with 
regard  to  test  length,  number  of  items  in  the  test  and 
output  uncertainties. 

In  the  following  jhaplor , a brief  discussion  is  pre- 
sented of  the  prior  research  which  has  been  conducted  in 
^his  area  by  the  RAND  Corporation  and  APINC  Research 
Corporation . 

Chapter  3 discusses  in  greater  detail  the  contractual 
provisions  which  are  important  in  this  study.  The  AFLC 
logistics  support  cost  model  is  explained  along  with  the 
rationale  for  the  abbreviated  model.  A brief  discussion 
of  relevant  versus  irrelevant  costs  is  included  in  support 
of  the  abbreviated  LSC  model. 

In  Chapter  4 the  simulation  model  employed  is  de- 
scribed in  sufficient  detail  so  as  to  be  understood  by 
those  who  are  not  acquainted  with  the  field  of  simulation. 
The  technical  details  of  the  simulation  model  along  with 
the  actual  computer  program  are  included  in  Appendix  D. 

A description  of  the  goodness  of  fit  tests  performed  on 
the  distribution  of  MLSC's  is  similarly  found  in  Chapter  4 
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with  the  technical  details  in  Appendix  D. 

A thorough  analysis  of  the  contributions  of  each  un- 
certain input  parameter  to  the  overall  uncertainty  is 
included  in  Chapter  4.  The  vehicle  for  the  analysis  is 
an  experimental  design  based  on  four  inputs:  MTBF,  MMH , 

"RTS,  and  test  length. 

Chapter  5 demonstrates  the  practical  applications  of 
the  simulation  methodology.  Included  are  three  applica- 
tions : 

1.  Determination  of  a suitable  COD  invocation  ratio 
and  test  length. 

2.  Analysis  of  a contractor  strategy. 

3.  Development  of  award  fee  design. 

Finally,  Chapter  6 includes  a summary  and  general- 
ization of  the  methodology,  conclusions,  and  recommenda- 
tions for  further  study.  A thorough  outline  is  presented 
for  a proposed  study  of  optimum  verification  test  length. 
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Chapter  2 

PRIOR  RESEARCH  INTO  LIFE  CYCLE  COST 
PREDICTION  AND  CORRECTION  OF 
DEFICIENCIES  PROVISIONS 

Introduction 

In  this  chapter,  a brief  discussion  of  the  important 
prior  research  in  this  area  is  presented.  The  purpose  of 
the  discussion  is  to  compare  the  assumptions  of  the  differ- 
ent approaches  and  demonstrate  the  consistency  in  results. 
This  discussion  should  serve  to  unify  the  various  independ- 
ent studies  which  have  been  conducted  in  this  field  to  date. 

Uncertainty  in  Life  Cycle  Cost 

The  primary  source  of  prior  research  in  this  area  of 
study  is  a working  paper  by  the  Rand  Corporation  (62).  The 
Rand  study  was  directed  toward  the  problems  of  confidence 
in  Life  Cycle  Cost  estimates  and  the  practicality  of  Life 
Cycle  Cost  models  as  aids  to  acquisition  decision  making. 

In  this  discussion  the  author,  E.S.  Timson,  compares 
the  uncertainty  of  Life  Cycle  Cost  estimations  at  various 
times  in  a program  development  cycle.  He  concludes  that 
the  uncertainties  in  Life  Cycle  Cost  prediction  during  the 
development  phase  are  so  great  as  to  make  the  AFLC  Life 
Cycle  Cost  Model  of  questionable  utility  in  supporting 
policy  decisions  such  as  enforcement  of  warranties  or 
establishment  of  operating  cost  targets. 
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The  paper  conducts  a case  study  of  the  K-16  inertial 
measurement  set  logistic  support  costs.  In  this  particular 
example,  there  is  an  extreme  disparity  he  tween  development 
phase  prediction  cf  LSC  and  that  which  was  measured  during 
the  operational  test  and  evaluation  (OT («£)  phase.  The 
predicted  LSC  was  $14,727  million.  The  measured  LSC  was 
$131,565  million. 

Timson  points  out  in  this  analysis  that  there  are 
three  basic  methods  for  establishing  the  uncertainty 
(variance)  of  the  measured  logistic  support  cost  as  a 
function  of  the  uncertainty  (variance)  of  the  input  random 
variables.  These  three  methods  are  convolution  integrals, 
Monte  Carlo  methods,  and  theory  of  errors.  He  continues  to 
state  that  convolution  integrals  are  difficult  to  use  unless 
there  is  a relatively  simple  relationship  between  the  input 
and  output  distribution.  Clearly  in  the  Life  Cycle  Cost 
Model  this  relationship  is  not  simple.  Monte  Carlo  methods, 
as  he  says,  can  accommodate  any  functional  relationship  be- 
tween input  and  output,  and  can  even  make  provisions  for 
dependent  inputs.  Error  theory,  which  Timson  uses  is 
adequate  for  a relatively  simple  functional  relationship 
where  the  input  random  variables  have  normal  distributions. 
The  error  theory  approach  to  this  problem  uses  the  following 
mathematical  relationship. 

The  variance  of  some  output  V(Y)  which  is  a function 
of  n independent  inputs  XJ....X  , is  given  by 
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V(Y)  - (||-)2  V(XX)  ♦ ...  (|y-)2  V(Xn) 

1 n 

As  will  be  shown  in  Chapter  4,  the  error  theory  approach 
breaks  down  in  the  F-16  application  due  to  the  mathematically 
intractable  stock  level  calculation  equations.  Some  question 
arises  also  as  to  the  propriety  o£  the  normality  assumption, 
particularly  with  regard  to  the  distribution  of  the  random 
variable  RTS  which  is  sampled  from  the  binomial.  Neither 
of  the  two  conditions  for  normality  are  met,  i.e.,  that  RTS 
Jbe  close  to  .5  and  the  number  of  samples  close  to  30. 

An  analysis  of  the  input  variables  used  in  Timson's 
analysis  is  very  interesting.  For  example,  for  the  input 
^distribution  of  MTBF,  a normal  distribution  is  specified 
with  a standard  deviation  equal  to  10  per  cent  of  the  mean. 

14o  information  is  available  with  which  to  make  a meaningful 
estimate  of  such  a distribution  of  MTBF ' s for  this  (P-16) 
application,  but  this  problem  is  avoided  by  using  the  actual 
distribution  of  failures  which  is  known  with  considerable 
confidence.  It  is  worthwhile  however,  to  attempt  to  approx- 
imate Timson's  result  using  F-16  data  to  determine  whether 
there  is  at  least  order  of  magnitude  agreement  between  the 
two  different  approaches  to  the  problem.  The  Timson  study 

is  based  on  a FLU  which  is  demonstrating  a 40  hour  MTBF 

MTBF 

during  OTiE.  Using  the  relationship  oMTBF  * where 

/n 

n * number  of  failures, an  F-16  FLU  with  a 40  hour  true  MTBF 
would  experience  approximately  88  failures  during  a 3500 
hour  test.  The  standard  deviation  for  the  mean  time 
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between  failures  then  would  be 


4 

o 1 


»,w  m 


or  , °^TBF  " 4,26 


^Thie  result  is  in  remarkably  close  agreement  with  Tiason's 
approximation.  Of  course,  without  knowlegde  of  the  length 
-of  the  OTfcE  he  is 'describing,  it  is  not  possible  to  deter- 
mine whether  this  apparent  consistency  is  real  or  coinci- 
dental t 

’ r>  It  is  also  possible  to  compare  the  distribution  of 
ATS  used  in  the  current  study  with  Timson’s  application. 

Once  again,  treating  the  event  reparable  this  station, 
not  reparable  this  station,  as  a Bernoulli  trial  the  events 
would  be  distributed  with  mean  n x RTS  and  variance 
A x RTS  x (1-RTS).  (n  is  the  number  of  failures)1  using 
n-20  and  RTS*. 95  we  have: 

' Mean  * 20  x .95  * 19 
■ Variance  * 20  x .95  x .05  » .95 
Standard  Deviation  * .975 

So  the  standard  deviation  here  is  5.13%  of  the  mean.  Timson 
has  ueed  10  per  cent  of  the  mean  for  the  standard  deviation 
of  his  normal  distribution  of  RTS. 

Overall,  it  would  appear  that  there  is  reasonable  con- 
sistency between  inputs  in  the  simulation  approach  and 
Tiason's  approech.  Some  consistency  can  also  be  seen  in 


■‘’Using  here  the  normality  assumption  which  as  stated  is 
questionable  in  this  application. 
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the  output  observations.  For  example,  Tims on  estimates  the 
standard  deviation  of  the  LSC  estimate  during  OT&E  to  be 
19  per  cent  of  the  mean.  The  current  analysis,  with  all 
three  random  variables  included  outputs  a standard  deviation 
of  10.3  per  cent  of  the  mean.  Driving  the  variance  of  MTBF 
to  sero  reduces  the  output  standard  deviation  of  the  Timson 
model  to  12  per  cent  of  the  mean.  A similar  change  in  this 
model  shows  an  output  standard  deviation  of  7.0  per  cent  of 
the  mean.  Finally,  by  making  all  variables  except  RTS 
deterministic,  the  Timson  model  shows  a standard  deviation 
of  5 per  cent  of  mean  while  this  model  shows  7.0  per  cent. 
Table  I below  summarizes  these  findings, 

TABLE  I 

ANALYSIS  OF  STANDARD  DEVIATION  TO  MEAN  RATIOS 


THIS  STUDY 

RAND  STUDY 

(13  PLUS) 

(1  FLU) 

(Per  cent  of  mean) 

(Per  cent  of  mean) 

All  inputs 

stochastic  10% 

19% 

°MTBP-0 

7% 

12% 

°M7BF-0 

/V 

7% 

5% 

°MMH  - 0 


It  is  apparent  that  Timson' s estimates  are  larger  than 
those  in  this  study  except  v>nen  0 ^mmh  • w^er®  they  are 
approximately  equal.  This  is  not  surprising  in  view  of  two 
facts  t 


1.  Timson' a study  addresses  only  one  FLU  which  results 
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in  a larger  overall  variance  to  mean  ratio. * 2.  Ti nson's 

study  includes  several  random  variables  not  included  in 
this  study.  (Six  random  variables  are  included  in  c5  which 
is  zero  in  this  application.) 

To  test  further  -for  consistency  between  the  two  ap- 
proaches , 'the  model  in  this  study  was  run  for  only  one  FLU 
with  all  three  factors  stochastically  input.  The  result  was 
a standard  deviation  of  29  per  cent  of  the  mean  which  is  in 
reasonably  close  agreement  with  the  19  per  cent  round  in 
the  Rand  study. 

The  Rand  study  concludes  with  the  comment  that  the 
deterministic  use  of  the  accounting  type  of  logistics 
support  cost  model  does  not  seem  appropriate.  It  further 
suggests  that  a stochastic  simulation  model  would  be  one 
possible  solution  to  meaningful  use  of  the  AFLC  LSC  model. 
This  suggestion  is  the  genesis  of  the  current  9tudy. 

Analysis  of  COD  Provisions 

An  exhaustive  study  of  the  development  and  analysis  of 
RIW  and  COD  provisions  for  the  air  combat  fighter  was  con- 
ducted by  the  ARINC  Research  Corporation  in  early  1975  (17). 
The  purpose  of  the  study  was  to  analyze  the  Life  Cycle  Cost 
controls  and  warranty  provisions  as  applied  to  the  Air 
Combat  Fighter  (ACF) , to  include  suggested  improvements 

2 

See  Appendix  A for  an  analysis  of  the  change  in  the  mean 
to  variance  ratio  of  KLSC  as  a function  of  the  number  of 
FLU’s  tested. 
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in  the  contract  provisions  as  well  as  an  evaluation  of  con- 
tractor responses.  As  such,  the  ARINC  study  provides  a 
stepping  off  point  for  this  analysis  and  deserves  a brief 
description. 

Of  particular  interest  is  the  ARINC  analysis  of  the 
correction  of  deficiencies  option.  At  the  point  in  tine 
when  the  ARINC  study  was  conducted.  General  Dynamics  had 
made  a proposal  regarding  a possible  reduction  of  their  COD 
bid  price.  The  initial  COD  bid  requirement  was  based  on  a 
correction  of  deficiencies  if  MLSC  was  greater  than  TLSC. 
General  Dynamics  offered  to  reduce  their  COD  bid  price  by 
$4,005,000  if  the  Air  Force  would  agree  not  to  impose  a 
COD  until  MLSC  exceeded  TLSC  by  more  than  15  per  cent.  By 
sensitivity  analysis  on  the  Air  Force  LSC  model,  ARINC 
determined  that  a reduction  of  average  MTBF  to  85  to  90 
per  cent  of  the  specified  value  would  result  in  an  MLSC 
equal  to  1.15  times  TLSC.  Further  analysis  showed  that  a 
25  per  cent  increase  in  TLSC  would  result  from  a reduction 
in  MTBF  to  about  77  to  80  per  cent  of  specified. 

ARINC  analyzed  the  uncertainty  in  the  MLSC  measurement 
in  the  following  manner.  Consider  a FLU  with  a true  MTBF 
of  175  hours.  During  a 3500  hour  test,  about  20  failures 
would  be  expected.  If  the  achieved  MTBF  were  only  77  per 
cent  of  specified,  about  26  failures  would  be  expected. 
ARINC  then  reasoned  that  because  of  the  uncertainties 
involved  in  the  verification  test  measurement  methods, 
and  the  fact  that  the  causes  of  sore  failures  are  certain 
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to  be  questionable,  there  would  not  be  a statistically 
significant  difference  between  20  and  26  failures  at  a 
reasonable  confidence  level.  Indeed,  after  consideration 
of  the  Rand  study  which  estimated  a 40  per  cent  "false 
pull"  rate,  it  is  apparent  the  results  of  the  verification 
test  could  be  swayed  in  either  direction  by  the  weight  of 

3 

"false  pulls"  (52:4).  The  estimate  of  a 40  per  cent 
false  pull  rate  is  confirmed  by  Balaban  (5:20). 

Using  the  methods  demonstrated  in  Chapter  1,  it  la 
possible  to  develop  a confidence  interval  about  an  MTBF 
derived  from  a given  test  length.  Using  the  above  example 
and  assuming  an  exponential  failure  rate: 


°MTBF  " 
(n-20) 
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°MTBF  mrnm 

(n-26) 

So,  a 70  per  cent  confidence  interval  for  MTBF  would  be: 
n-20  (136-214) 

n-26  (108-160) 

Since  the  confidence  intervals  overlap  at  a relatively  low 
confidence  level  of  70  per  cent,  it  seems  clear  that  a 
difference  between  20  and  26  failures  would  not  be  a 
sufficiently  reliable  distinction.  The  conclusion  which 


A false  pull  is  defined  here  as  a maintenance  removal 
which  was  subsequently  checked  and  found  to  be  operating 
correctly. 
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holds  hsrs  for  only  a single  PLU  with  one  source  of  un- 
certainty is  generalized  to  many  PLUS  and  several  sources 
of  uncertainty  in  the  current  analysis* 

AH  INC  recommended  the  following » If  General  Dynamics 
would  continue  to  reduce  their  COD  price  at  the  rate  offered 
($4.0  million  for  a IS  per  cent  no  fault  zone*  or  $267*000 
for  each  no  fault  percentage  above  TLSC) * the  Air  Force 
should  agree  to  a 25  per  cent  no  fault  zone  above  the  TLSC 
and  a reduction  in  the  original  COD  price.  This  was 
accomplished.  In  a further  analysis  of  this  particular 
issue  this  study  will  show  how  a given  amount  of  certainty 
can  be  provided  to  the  contractor  by  increasing  test  length 
rather  than  necessarily  relaxing  the  constraint  upon  the 
contractor  as  was  done  in  the  F-16  contract.  For  example, 
it  can  be  shown  that  under  the  original  F-16  COD  provision, 
wherein  the  COD  would  be  invoked  if  MLSC  was  greater  than 
TLSC,  the  contractor  is  exposed  to  a 50  per  cent  chance  of 
an  incorrect  invocation  of  COD.  Under  the  provision  that 
COD  would  not  be  invoked  until  MLSC  was  greater  than  1.25 
times  COD,  the  contractor*  risk  of  erroneous  invocation  of 
COD  is  only  about  2 per  cent.  If  the  amount  of  risk  which 
the  contractor  is  prepared  to  take  can  be  established  in 
this  manner,  then  the  Air  Force  can  provide  that  risk  level 
to  the  contractor  in  two  ways.  The  first  is  as  above, 
adjustment  of  the  invocation  point.  The  second  is  adjust- 
ment of  test  length.  The  actual  methodology  for  the 
determination  of  test  length  is  given  in  Chapter  5. 
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The  ARINC  study  reviewed  the  experience  of  the  Air 
Force  in  RIW/COD  contracting  and,  among  their  recommendations 
was  this:  "Establish  and  specify  the  criteria  to  be  used 

for  determining  the  amount  of  the  award  fee  that  the  con~ 
tractor  will  receive  if  the  MLSC  ir*  equal  to  or  less  than 
TLSC."  This  recommendation  provides  one  of  the  stimuli 
for  the  current  study. 
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Chapter  3 

CONTRACT  PROVISIONS  AND  THE  LOGISTIC 
SUPPORT  COST  MODEL 


Introduction 

In  this  chapter  the  important  contractual  provisions 
which  are  critical  to  this  analysis  are  discussed  in 
sufficient  detail  to  provide  a thorough  understanding 
of  the  correction  of  deficiencies  clauses  and  the  pro- 
cedures for  the  verification  test. 

A summary  of  the  AFLC  logistics  support  cost  model  is 
also  presented.  Each  term  is  explained  along  with  a dis- 
cussion of  the  reasoning  and  assumptions  involved  in  in- 
cluding or  excluding  it  from  the  abbreviated  logistic 
support  cost  model  employed  for  a FLU  level  analysis. 

Contractual  Provisions  for 
Correction  of  Deficiencies 
and  Supportability 
Verification  Test 

There  is  a total  of  $8,400,000  in  award  fees  poten- 
tially available  to  the  contractor.  As  previously  stated, 
this  is  divided  into  two  separate  award  fees  of  $2,000,000 
and  $6,400,000. 

The  $2,000,000  fee  is  based  on  the  logistic  support- 
ability  of  the  control  FLUs.  There  will  be  either  12  or 
13  control  FLUs  depending  on  which  radar  contractor  is 
ultimately  chosen.  The  control  FLUs  for  both  potential 
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radar  suppliers  are  shown  in  Table  II. 

It  can  be  seen  from  Table  II  that  there  wn  i be  13 
FLU's  if  Hughes  radar  is  purchased,  or  12  FL'J's  if 
Westinghou9e  radar  is  purchased.  In  order  to  provide  a 
vehicle  for  the  methodology  without  duplicating  every 
step,  the  Hughes  radar  was  chosen  for  this  analycis  based 
on  discussion  with  AFLC  AQMLA  (2).  The  function  of  the 
model  and  methodology  is  identical  for  either  radar  and  the 
reader  can  readily  reconstruct  the  methodology  for  the 
Westinghouse  radar  if  desired. 

The  contract  provides  for  changes  to  the  target 
logistic  support  cost  of  control  FLUs  (TLSC-COD)  the 

target  logistic  support  cost  of  non  control  FLUs  (TLSC- 
System)  in  six  specific  cases.  There  are 

1.  Approved  Engineering  Change  Proposals  (ECP’u)  in 
conjunction  with  individual  renegotiated  values  resulting 
from  the  engineering  change. 

2.  Changes  in  the  anticipated  force  structure  of 
activity  levels  to  be  supported. 

3.  Inflation  factor  adjustments  to  acquisition  cost 
elements. 

4.  Changes  to  factors  defining  the  maintenance 
concept  resulting  from  a government  approved  repair 
level  analysis. 

5.  Adjustment  due  to  selection  of  radar. 

6.  Adjustment  due  to  subsequent  identification  of 
certain  control  FLUs  designated  as  government  furnished 
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FLU 

Headup  Display 
Navigation  Unit 
Fire  Control  Computer 
Electronics  HUD 
Flight  Control  Computer 
Radar  EO  Display 
Digital  Scan  Converter 
Electronics  EO  Display 


TABLE  II 
LIST  OF  FLU'S 

WORK  UNIT  CODE  (WUC) 
74  BAO 
74  DAO 
74  CAO 
74  BCO 
114  ABO 
74  EAO 
74  ECO 
74  EBO 


WESTINGHOUSE  RADAR 


Antenna  Servo  74  AAO 
Low  Power  RF  74  ABO 
Digital  Processor  74  AEO 
Transmitter  74  ACO 

HUGHES  RADAR 

Receiver  Exciter  74  ADO 
Data  Processor  74  AEO 
Signal  Processor  74  ADO 
Transmitter  74  ACO 
Antenna  74  AAO 


Note:  Detailed  Specifications  for  each  FLU  are  found  in 

Appendix  B. 


V 

r 

* 

i 

f j 


i j 
* 


42 


GOR/SM/75D-6 


equipment  (GFE) . 

The  above  are  the  only  allowable  adjustments  to  TLSC.  It 
is  important  to  note  then  that  inflation  is  essentially  a 
"flow  through,"  or  an  exogenous  variable.  The  provision 
which  allows  for  adjustments  due  to  changes  in  activity 
level  is  necessary  because,  as  will  be  shown,  the  LSC 
equations  are  driven  by  the  terms  peak  force  flying  hours 
<PFFH)  and  total  force  flying  hours  (TFFH) . It  will  be- 
come apparent  in  the  analysis  that  a change  in  TFFH  or 
PFFH  will  not  affect  the  overall  probability  distribution 
of  MLSC's.  This  observation  is  confirmed  by  the  ARINC 
study  (17: A- 13). 

The  contract  clearly  states  that  the  values  of  MTBF 
will  not  be  renegotiated,  and  further,  that  any  changes 
to  organizational,  intermediate,  or  depot  level  man  hour 
values  shall  retain  the  same  gross  weighted  manhour  cost 
value  (man  hours  expended  times  labor  rate) . Symbolically, 
this  could  be  stated: 

If  (PAMH  X BLR  + IMH  X BLR  + BMH  X BLR  + DMH  X DLR)  - K 
Then  PAMH.  X BLR  + IMH.  X BLR  + BMH,  X BLR  * DMH,  X DLR)  = K 

x i.  l i 

BLR:  Base  labor  rate 

DLR:  Depot  labor  rate 

In  other  words,  the  workload  can  be  shifted  among  the  organi- 
zational levels,  but  no  negotiated  change  in  the  aggregate 
labor  cost  per  FLU  is  allowed. 

The  Verification  Test 

The  verification  test  to  collect  data  for  the  purpose 
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of  measuring  the  contractor's  eligibility  to  receive  the 
award  fees  will  be  conducted  by  the  Air  Force  using  the 
first  operational  combat  crew  training  squadron  (CCTS) . 
The  test  will  begin  six  months  after  activation  of  that 
squadron,  it  is  assumed  then,  that  by  the  sixth  month 
after  activation,  thp  problems  of  infant  mortalities  in 
avionics  equipment  should  be  over.  This  assumption  is 
necessary  to  the  use  of  the  exponential  distribution  for 
generation  of  failures.  If  a test  were  to  be  conducted 
during  a period  when  infant  mortalities  were  being 
experiences  it  is  likely  that  a much  higher  than  spec- 
ified failure  rate  would  be  observed.  It  would  be  nec- 
essary to  use  a Weibull  distribution  of  failures  with  a 
shape  parameter  m<l  to  represent  such  a process.  See 
Figure  4. 


If  in  the  actual  verification  test  it  can  be  shown  that  the 
distribution  of  failures  for  a given  FLU  is  represented  by 
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the  Weibull  distribution  with  m<l,  then  this  observation, 
combined  with  s low  MTBF  would  be  symptomatic  of  a defi- 
ciency in  equipment  burn  in.1 

Since  the  procurement  document  does  not  eddrese  the 
maintenance  and  operations  learning  process,  it  has  been 
assumed  as  stated  in  Chapter  1,  that  this  learning  process 
is  not  a significant  factor.  ARINC  has  recommended  that  the 
Maintenance  and  operations  personnel  who  participate  in  the 
six  month  test  be,  to  the  extent  practicable,  personnel 
who  have  acquired  some  experience  with  the  equipment  prior 
to  the  test.  This  recommendation,  although  well  intentioned, 
will  not  obviate  the  problems  of  inexperience  for  two  rea- 
sons. a.  Not  all  maintenance  and  operations  personnel 
will  have  the  desired  six  months  of  experience,  b.  Six 
months  of  operational  experience  is  not  sufficient  to  reduce 
operator  error  to  its  final  steady  level.  The  term  oper- 
ator error  is  used  here  primarily  to  describe  incorrect 
operator  diagnosis  which  in  turn  impacts  upon  the  "false 
pull*  rate  and  ultimately  upon  the  MTBF. 

It  is  important  to  realize  then  that  this  learning 
process  has  been  assumed  away  in  this  analysis  and  is  not 
represented  in  the  simulation  model.  If  sufficient  data 
were  available  to  describe  the  process  it  could  be  in- 
corporated into  the  model.  The  relative  rate  at  which 

^The  term  "burn  in"  is  commonly  used  as  the  equivalent  of 
the  term  "break  in"  when  speaking  of  avionics  equipment. 
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technical  learning  occurs  has  been  estimated  by  M.A.  Wilson 


(65:435)  as  shown  in  Figure  5 below. 


In  order  to  incorporate  such  a model  into  the  method- 
ology given  here,  it  would  be  necessary  to  know  only  the 
position  on  the  abscissa  for  the  average  technician  for  a 
given  task.  As  can  be  seen,  the  curve  flattens  out  after 
15  to  20  repetitions  so  that  learning  effects  beyond  this 
point  may  be  considered  negligible.  The  curve  which  Wilson 
derived  above  was  for  laboratory  learning  conditions,  and 
it  would  be  expected  that  field  conditions  would  vary  some- 
what from  these. 

Regarding  this  aspect  of  the  contract,  ARINC  has  made 
the  following  observation. 

The  contractor  will  be  inclined  to  contest  a declared 
deficiency  centering  on  man  hours  required  to  perform 
maintenance  due  to  inexperience  or  errors  on  the  part 
of  Air  Force  maintenance  technicians.  Ultimately,..., 
deficiency  of  man  hour  expenditure  must  be  resolved 
either  to  a specific  deficiency  of  the  contractor's 
test  procedures,  test  equipment,  etc.  or  to  an  en^r 
or  lack  of  experience  on  the  part  of  Air  Force  main- 
tenance (17:A-10J. 


i 


\ 
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The  government  will  prepare  a detailed  test  plan  to 
assure  that  all  data  necessary  to  compute  the  appropriate 
support  costa  will  be  collected.  The  government  will  be 
responsible  for  ell  organizational,  intermediate,  and 
depot  level  maintenance  and  supply  support  for  the  veri- 
fication test. 

Provision  is  made  in  the  contract  for  retesting  in 
case  correction  of  deficiencies  action  is  taken.  That  is, 
following  the  correction  of  deficiencies,  the  government 
intends  to  verify,  through  such  additional  testing  as  it 
may  deem  necessary,  that  the  TLSC-COD  has  been  achieved 
for  the  control  FLUs  selected  for  COD  coverage.  The  con- 
tractor obligation  under  this  provision  shall  continue 
untill  satisfactory  compliance  is  demonstrated. 

During  the  verification  test,  the  contractor  will  pro- 
vide representatives  to  verify  the  authenticity  of  the 
observed  data.  Representatives  will  also  be  provided 
for  a rete3t  should  one  become  necessary.  In  the  event 
a retest  is  required,  the  price  for  that  portion  of  the 
test  which  is  conducted  in  the  contractor's  facility  will 
be  separately  negotiated. 

Those  items  of  the  LSC  model  which  were  subject  to 
verification  were  listed  and  defined  in  Chapter  1.  They 
are  listed  again  in  Table  ill  together  with  their  appro- 
priate measurement  methods. 
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FACTOR 
N,  QP A,  K 
UC,  CAB,  CAD 

MTBP 


RIP,  RTS,  NRTS,  COND 


TABLE  III 

MEASUREMENT  METHODS 

MEASUREMENT  METHOD 
Direct  observation 

The  average  negotiated  unit  prices  in 
effect  at  the  tine  of  the  teat. 

The  total  reported  flying  tine  during  the 
test  period  times  the  QPA  divided  by  the 
number  of  failures  for  each  FLU.  For 
failure  definition  see  Note  3 below. 

It  is  to  be  noted  that  the  MTBP  established 
in  this  manner  is  expressed  in  flying  hours. 
As  such,  the  value  of  1/MIBP  defined  in  this 
manner  will  be  analogous  to  the  expression 
UF/MTBF  which  was  originally  predicted.  No 
UF  factor  will  be  established  directly.  (UP 
is  ratio  of  operating  hours  to  flying  hours. 

Observed  fraction  of  total  failures  repaired 
in  place,  in  baselevel  shops,  at  the  depot, 
or  condemned  at  base  respectively,  as 
averaged  over  the  test  period. 


(ALL  MAINTENANCE  NAN  HOUR 
VALUES  ASSOCIATED  WITH  FLU 
AND  SYSTEM  LEVEL  MAINTENANCE 
PAMH,  INK,  RMH,  BMt,  DM1) 


Reported  man  hour  expenditures  against 
appropriate  when  discovered,  how- 
malfunctioned  and  action  taken  codes 
(IAW  AFM  300-4).  The  recorded  values 
will  be  averaged  over  the  test  period.) 


DOWN 


Reported  down  time  for  peculiar  support 
equipment  (Average  over  test  period) . 


APPLICABLE  TO  TLSC  SYSTEM  ONLY 


FLA,  ba  The  average  negotiated  costs  for  equipment 

necessary  to  support  each  squadron  and  base 
level  8 hop  respectively. 

SMI  The  scheduled  maintenance  interval  prescribed 

by  the  appropriate  technical  order. 

Note  1:  Only  the  first  13  parameters  are  used  in  the  control  FLU 

LSC  model. 

Note  2 s FLUs  subject  to  verification  will  be  those  installed  in  the 
production  aircraft  delivered  to  the  combat  crew  training 
squadron  as  well  as  any  replacement  spares  delivered  to  support 
supply  and  maintenance  requirements.  These  components  will 
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undergo  normal  acceptance  testing.  So  extraordinary  qualifi- 
cation testing  will  be  authorised.  All  organizational  and 

intermediate  maintenance  shall  bo  performed  by  Air  Force  using  V 

cosetand  personnel.  Depot  maintenance  will  be  performed  by  the 
su.  • •••  designated  air  logistics  center  to  the  maximum  extent  jpracti- 
cabie.  For  control  FLUs  under  COD  the  contractor  is  allowed 
‘ the  right  for- inepaction  for  all  government  Identified  failures. 

For  COD  control  FLUs,  no  maintenance  at  base  level  will  be 
Vt'  performed  except  removal  and  replacement  until  a contractor 
representative  is  present,  provided  he  is  present  within  24 
'<  -r  > fcodrs  of  notification.  This  provision  c|oes  not  apply  to  noh 
control  FLUs. 

Hots  3t  The  definition  of  a failure  will  be  consistent  with  that  used 
•j0  for  reporting  and  consolidation  under  the  AFM  66-1  maintenance 
data  collection  system.  A failure  shall  be  considered  as  any 
departure  from  the  required  performance  in  excess  of  the 
allowable  tolerances  defined  in  the  appropriate  configuration 
•ini.  ‘ item -specification. 

It  should  be  noted  here  that  a removal  which  is  found  to 
wav  . i be  aervi gamble  at  bench  check  or  depot  shall  still  be  deemed 
a failure  if  the  erroneous  failure  identification  is  due  to 
inadequately  described  test  procedures  or  test  equipment 
developed,  procured,  or  prescribed  by  the  contractor. 

Failures  caused  by  fire,  explosion,  or  aircraft  crash 
are  exempt. 


Sumnary  of  AFLC  Logistics 
Support  Cost  Model  and 
Discussion  ofi  Relevant  Costs 

The  'general  approach  in  this  section  is  to  attack  the 
AFLC  logistic  support  cost  model  on  a term  by  term  basis.. 

An  explanation  of  variable  names  is  included  as  Appendix  F. 
As  each  term  of  the  model  is  presented,  an  explanation  of 
why  it  is  or  is  not  included  in  the  TLSC-COD  and  TLSC- 
Bystem  computations  is  offered.  For  those  costs  whose 
inclusion  is  sublet  to  controversy  an  expanded  discussion 
of  relsvance  is  presented. 

The  computerized  AFLC  logistic  support  cost  model 
consists  of  ten  equations,  each  of  which  describes  a 
portion  of  the  resources  required  for  an  operating  logistics 
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system.  The  model  provides  a method  for  estimation  of  the 
expected  support  costs  which  may  be  incurred  by  adopting  a 
particular  system.  It  is  used  to  compare  and  discriminate 
among  design  alternatives  where  relative  cost  difference  is 
the  desired  measure  of  merit.  The  significant  result  then 
is  not  the  absolute  value  of  the  output  but  rather  the 
difference  between  competing  alternatives. 

The  following  assumptions  which  are  fundamental  to  the 
model  must  be  considered  in  using  the  results. 

1.  The  model  assumes  a uniform  level  of  activity  at 
each  base. 

2.  The  stock  level  for  spares  and  pipeline  quantities 
are  computed  to  support  the  peak  level  of  program  activity. 

No  provision  is  made  for  incremental  build  up. 

3.  The  model  explicitly  computes  only  those  logistic 
support  costs  associated  with  the  weapon  system,  system,  and 
first  line  units.  Components  below  FLU  level  are  considered 
only  implicitly. 

4.  The  model  assumes  one  depot  repair  location  and 
any  given  number  of  imtermediate  level  repair  locations. 

5.  Quantities  of  support  equipment  are  based  on  a 
manhour /machine  hour  equivalence.  In  other  words,  the  model 
assumes  that  a given  piece  of  support  equipment  is  in  use 
during  the  entire  elapsed  time  period  over  which  labor  (in 
manhours)  is  required  to  perform  a task. 

6.  Certain  elements  of  resource  consumption  for  which 
there  is  no  basis  for  estimation,  are  not  included.  Examples 
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are, .modification  costs,  and  cost  of  maintenance  *ct ions 
generated  by  false  pulls. 

The  .firpt  eight  equations  of  the  LSC  model  .are  struc- 
tured tp  aggregate  the  cost  of  each  system  within  the  weapon 
system  including  subordinate  flus  and  support  equipment. 
Equations  9 and  10  compute  propulsion  system  goats.  The 
ten  equations  are  described  below. 


1.  SPARES 


The  first  equation,  C.,  is  the  cost  of  spares  (initial 
and  replacement) . 


(3-1) 


1 cost  of  spare  FLUs 
n n 

:MZ  (STK. )(UC.)  + Z (PFFH) (QPA. ) (UF. ) (1-RIP. ) 
. i-i  1 1 i-1  11  1 

• (NRTSi)  (DRCTi ) (UC^ 


1 


n 

+ Z 
i«l 


MTBFi 


(TFFH) (QPA. ) (UF . ) (1-RIP . ) (COND  ) (UC. ) 

IX  X XX 

MTBFi 


ak: 


an 


i' 


, She  first  term  in  (3-1)  is  the  investment  in  base  stock, 
e.-:  . The  second  term  is  depot  repair  pipeline  stock.  This 
second  term  is  essentially  the  peak  number  of  failures  per 
month,  (PFFH/MTBF;,  times  the  fraction  which  are  shipped  to 
depqt  .time?  the  depot  repair  cycle  time.  This  equation 
Obviously  will  provide  for  an  overabundance  of  pipeline 
>3P area  ih  those  months  when  flying  is  conducted  at  less 
than  peak  level. 

The  third  term  represents  the  total  number  of  condem- 
- nations  which  will  be  replaced  over  a total  force  flying 
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program  of  2,411,130  hours. 

The  entire  Cl  equation  is  applicable  to  both  TLSC 
system  and  TLSC-COD.  The  fraction  of  failures  condemned 
iCOND)  however  is  tero  for  all  control  PLUS  which  effec- 
tively eliminates  it  from  the  equation. 

The  computation  of  STK,  for  the  first  term  involves 
the  computation  of  a mean  demand  rate  per  base  where 


X ■ 


(PFFH) (QPA) (UFi) (l-RIPi) 

m^tbfT 


So  Lamda  represents  the  average  demand  per  base  during 
a peak  flying  level  month.  Next,  a weighted  pipeline  time 
must  be  computed.  That  is: 

tL  = (RTSa)  (BRCT^-MNRTS^  l(OSTCON)  (l-OS)  + (OSTOS)  (OS)  ] 
This  is  the  average  amount  of  time  at  base  level  to  cycle  an 
item  from  failure  back  to  serviceability  and  installation. 
The  second  part  of  the  term  provides  for  overseas  shipping 
of  an  appropriate  fraction  of  the  items. 


So  the  product  of  and  is: 


(X 


i month^  ^i  raonths  pipeline  time)  *(  Demands  over  average! 


FLUs 


[Expected  No.  of 


(Repair  Pipeline  TimeJ 
The  actual  stock  level  STK  is  established  by  the  follow- 
ing inequality. 

Find  the  minimum  value  of  STTT  such  that 


x>STKJ 


(x-STKi)P(x| Xiti) <EBO, 


1 


In  all  equations. 


the  subscript  i represents  the  ith  FLU. 
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where  EBO  ir.  a government  established  acceptable  expected 
back  order  level  and  the  distribution  of  probabilities  of 
demand  is  a Poisson  distribution  with  mean  (X^t^).  When 
the  equation  io  solved  for  the  minimum  value  of  STK^,  then 
the  cost  for  the  ith  FLU  at  all  bases  is  (M)  • (STK^)  • {UC^  « 
The  Poisson  distribution#  Figure  6 below#  is  quite 
commonly  used  to  represent  inventory  demand  rates.  Its  use 
required  only  the  assumptions  that:  1.  The  likelihood  of 

the  occurrence  of  a demand  in  a given  period  does  not  change 
over  time.  2.  That  the  occurrence  of  an  event  has  no 
effect  on  whether  or  not  a subsequent  event  occurs  (.11:38). 


The  expected  back  order  (EBO)  term  is  a constant  which 
is  determined  by  Air  Force  policy.  It  is  of  interest,  how- 
ever, to  examine  the  behavior  of  logistic  support  cost  as  a 
function  of  changes  in  EBO.  To  this  end,  a sensitivity 
analysis  has  been  performed  on  EBO  as  is  shown  in  Appendix  C. 
This  sensitivity  analysis  shows  that  for  values  of  EBO  less 
than  approximately  .05,  the  logistic  support  cost  increases 
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V. 1 rapidly.  For  such  low  values  of  EBO,  a large  number  of 

spares  wouxd  be  required.  In  other  words,  it  would  become 
very  expensive  to  reduce  the  risk  of  shortage  much  below 
.05.  Decreasing  EBO  from  .1  to  .05  would  cost  about 
$2.6  million,  while  decreasing  it  from  .05  to  .03  would 
cost  about  $3.0  million. 

Increasing  the  value  of  EBO  above  . 1 and  accepting 
greater  risks  of  sho» tage  would  have  very  little  effect 
on  logistic  support  cost.  For  example,  increasing  EBO 
to  .2  would  save  about  $2.4  million.  A further  increase 
from  .2  to  .3  would  save  only  about  $2.0  million. 

In  summary,  small  reductions  in  EBO  (down  to  .05) 
could  be  made  at  a relatively  modest  cost,  while  further 
reductions  would  soon  become  prohibitively  expensive. 

Fairly  large  increases  in  EBO  are  required  to  obtain 
substantial  decreases  in  logistic  support  cost.  The 
value  of  .1  used  in  this  analysis  seems  well  chosen,  though 
it  would  be  reasonable  to  consider  reductions  in  the  range 
of  .05  to  . 1 . 

2.  ON  EQUIPMENT  MAINTENANCE 

The  second  equation  in  the  LSC  model,  C2,  represents 
the  cost  of  on  equipment  maintenance,  i.e.. 
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(3-2)  Cj  * Cost  of  on  equipment  maintenance J 
n (TFFH) (QPA.) (UF . ) 

’ 1 MTBF  ~ ♦ (RIPj)  (IMH.) 

i»l  1 

+ ( 1-RTP  ■ ) (RMH  ) ] (BLR)  + (SMH)  (BLR) 

1 1 SMI 

l « 

* ■ (TFFH) (FPA) (ERMH) (BLR)  • 

: cron : 

» — .-j 

The  first  term  im  (3-2)  is  the  expected  cost  of  un- 
scheduled in  place  maintenance  over  the  lifetime  of  the 
equipment.  It  includes  allowance  for  manhours  expended 
in  preparation  and  access  activities,  PAMH,  actual  in  place 
maintenance  activities  (IMH) , and  removal  or  replacement 
activities  (RMH) . 

The  second  term  provides  for  scheduiv.d  maintenance 
manhour  costs  over  the  equipment  lifetime. 

The  third  term  applies  only  to  power  plant  systems. 

It  accounts  for  the  maintenance  manhours  required  for 
removal  and  replacement  of  engines  over  the  total  force 
flying  hours. 

In  the  present  application,  the  first  term  of  3-2 
is  applicable  to  both  TLSC-system  and  TLSC-COD.  The 
second  term  is  applicable  to  TLSC-system  only.  The  third 
term,  engine  maintenance,  is  not  used  here  as  it  is  not 
applicable. 

All  engine  related  terms  will  be  enclosed  in  dotted  lines 
as  above.  These  terms  are  not  relevant  to  this  analysis. 
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Serious  questions  have  been  raised  regarding  the  relevance 
of  the  cost  element  to  a decision  between  competing 
alternatives  or  to  an  application  involving  measurement  of 
logistics  supportability . Consider  what  3-2  is  measuring. 

The  equation  multiplies  the  aggregate  manhour  expenditure 
-by  the  base  labor  rate  to  determine  an  overall  cost.  This 
. base  labor  rate  is  a summation  of  the  estimated  hourly  wage 

- gvf- 

^ of  an  average  Air  Force  skill  level  plus  the  pro  rata  share 
of  base  level  costs  requi  Jd  to  support  that  technician. 

It  is  important  to  emphasize  at  this  point  that  a life  cycle 
cost  measurement  system  ought  to  exclude  any  fixed  costs 
which  would  be  incurred  whether  a particular  system  is 
procured  and  operat  d or  not  (24)  . If  a particular  cost 
will  be  incurred  regardless  of  what  choice  is  made  between 
alternatives  or  regardless  of  whether  or  not  a system  is 
bought  and  operated,  then  that  cost  is  irrelevant  to  any 
life  cycle  cost  decision  making,  and  it  should  be  excluded 
from  the  model.  It  could  be  argued  that  for  policy  or 
political  reasons  many  base  level  costs  would  continue 
even  if  a new  system  such  as  the  F-16  were  not  procured  and 
operated  at  those  bases.  It  could  also  be  argued  that  a 
given  reduction  in  manhour  requirements  for  a weapon  system 
does  not  always  result  in  the  same  proportional  reduction 
in  unit  manning  levels.  These  manning  levels  may  be  deter- 
mined more  directly  by  policy  and  contingency  plans  than  by 
current  requirements.  These  are  very  difficult  questions 
and  good  arguments  can  be  made  for  both  sides.  One  important 
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point  is  offered  here  in  support  of  the  relevance  of  these 

costs. 

If  the  assertion  that  manning  levels  are  determined  by 
policy  rather  than  actual  hardware  reauirements  is  accepted 
at  face  value,  then  the  next  logical  Question  is:  How  are 

the  policies  for  manning  levels  established?  It  i3  reasonable 
to  assume  that  these  manning  levels  would  be  established  in 
response  to  some  contingency  plan  which  required  a work  force 
in  excess  of  that  required  for  routine  operations  and  perhaps 
even  in  excess  of  that  reqr  to  support  the  peak  force 
flying  hours.  Further,  i._  „ts  safe  to  assume  that  there 
is  some  relationship  between  policy  ostablj  i liec  manning 
levels  and  actual  hardware  man  hour  requirements.  in  this 
light  then,  even  though  a unit  might  have  an  authorized  to 
required  ratio  of  say  1.25/1  it  would  be  expected  that,  in 
the  long  run,  as  it  becomes  apparent  that  improvements  in 
reliability  and  maintainability  have  truly  occurred,  the 
policy  makers  would  reduce  the  ratio  somewhat.  This  theory 
is  particularly  believable  in  that  there  is  general  recogni- 
tion of  the  fact  that  manpower  costs  account  for  a very 
large  portion  of  the  DOD  budget.  In  the  short  run,  the 
ratio  could  be  reduced  by  retraining  of  excess  technicians 
into  weapons  systems  which  have  shortages.  In  the  long  run, 
the  ratio  would  be  reduced  by  cutbacks  in  recruiting.  In 
summary,  it  is  probably  true  that  manning  levels  are  not 
directly  established  by  weapons  systems  manhour  require- 
ments, but  at  the  same  time  it  is  reasonable  to  expect  that 
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in  the  long  run  the  manning  levels  will  be  responsive  to 
requirements.  The  conclusion  offered  here,  then,  is  that 
the  manhour  costs,  together  with  the  pro  rata  share  of  base 
services  are  a relevant  cost  in  life  cycle  cost  calculations. 


3.  OF**  '.'QU1PMENT  MAINTENANCE 

The  third  equation  in  the  model  represents  off  equip-* 
Stent  maintenance  costs,  i.e., 

(3-3)  C3  = cost  of  off  equipment  maintenance 


n (TFFH) (QPA . ) (UF . ) (1-RIP . ) 


MTBFi 


— { (BCMFT)  (BLR) 


+ RTSi  ( (EMIT)  (BLR  + BMR)  + (BMC.)(UCiU 
+ (2(NRTSi)  + CONDJ I (PSC) (1-OS) 


+ (PSO) (OS)} (1.35  Wi) } 


+ ; (TFFH) (CPA) (1-ERTS) (EOH) (EUC)  , 
. CMRI 

L. J 


The  first  term  in  3-3  describes  the  cost  of  main- 
tenance for  those  FLU  failures  which  are  not  repaired  in 
place.  This  term  is  applicable  to  both  TLSC-systero  and 
: - .0^.  For  those  failures  which  are  reparable  on  station 

on  the  equation  finds  the  sum  of  the  costs  for  bench  check- 
ing (BCMH)  and  for  direct  repair  manhours  (BMH) . Also  in- 
cluded are  the  implied  cost  of  stock  and  repair  of  lower 
level  sub  assemblies  (BMC) . Costs  incurred  at  depot  level 
are  summed  in  the  same  equation  in  an  exactly  parallel 
fashion.  Also  included  are  the  costs  of  shipping  and  depot 
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j. 

I 


level  condemnations.  The  last  tern  applies  only  to  power- 
plant  systems  and  is  not  included  in  this  analysis. 

The  argument  for  relevance  of  manhour  costs  In  this 
equation  is  exactly  the  same  as  that  for  on  equipment  main- 
tenance, and  it  will  not  toe  restated  here. 

4 . INVENTORY  MANAGEMENT 

The  fourth  equation  is  inventory  management  cost,  i.e., 
(3-4)  = Inventory  management  cost 

n 

- (IMC  + (PIUP)  (RMC)  ) Z (PA.  + PP.  +1) 

i=l  1 1 

n 

+ (M) (SA) (PIUP)  L (PA.  + PP.  + SP.  +1) 
i-1  1 x 1 

The  first  term  in  3-4  is  the  cost  for  entering  an  item 
into  inventory  and  maintaining  it  over  the  system  life.  The 
second  term  accounts  for  the  base  level  inventory  costs. 

This  equation  is  excluded  from  tooth  TLSC-system  and 
TLSC-COD  calculations.  It  is  considered  a negligible  cost. 

The  same  general  questions  of  relevance  arise  with 
tnis  o-.'.uatior.  as  with  and  C3-  No  detailed  discussion  is 
offered  here  since  the  equation  is  not  used  in  this  appli- 
cation It  is  sufficient  to  say,  that  in  any  application, 
the  relevance  of  the  cost  ought  to  be  evaluated  in  terms  of 
whether  it  is  a cost  which  can  be  avoided  if  the  system 
is  not  procured  and  operated.  This  evaluation  should  con- 
sider the  long  run  as  well  as  the  short  run. 
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5 . SUPPORT  EQUIPMENT 

Equation  five  is  the  cost  of  support  equipment,  i.e., 
(3-5)  C-  “ Cost  of  support  equipment. 


• -r 


n (PFFH)  (QPA^U-RIP^  k (RTSj.)  (BMHj+BCMH^ 
z mtbfT  j * 1 TBilip  (BAA)  (1-Down^) 

(NRTS.) (DMH . ) 
U+(PiuP)(coB.)]CABj  ♦ <DURJ(Dfrytl“ 

-Dcwn^) 

[l-t(PIUP)  (CODj)  )CADj  ♦ (l+.KPIUP)  ] (DCA+DPA 


+ M (BCA+BPA+FLA)  ] + CS  + IH 

The  first  term  in  3-5  calculates  the  acquisition  and 
maintenance  costs  for  that  support  equipment  which  is  re- 
quired for  support  of  FLUs.  Workload,  usage  rate  of  support 
equipment,  and  down  time  of  support  equipment  are  all  con- 
sidered in  this  calculation. 

The  second  term  accounts  for  support  equipment  which 
is  non-workload  related. 

The  cost  element  C5,  is  zero  for  all  control  FLUs 
since  there  is  no  peculiar  item  of  support  equipment  re- 
quired for  any  control  FLU.  Equation  C,.  is  present  in 
TLSC-system  calculations. 


6.  PERSONNEL  TRAINING 

Equation  six  is  the  cost  of  personnel  training  i.e.. 
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(3-8)  Cg  * cost  of  personnel  training, 


tl-HPlUP)  (TRB)  ] TCB  f y 
(PIUP)  (PMB) 


(TFFH)OPAi) (UF. ; 
MTBF~ 


1 PAMU  . + !;ur  1 (IMH  ) + ( 1-RIP  . ) [ RMH  +BCMII . + (RTS  . ) (BMH  . )> 
(All  11X1  i'j 


. TFrii  : TFFH)  (EPA)  { ERMH)  • 

♦ §Hjr  (SMH)  * : cmri 


— . .i 


t U(lIOl-l)ITRD)  „D  ? ITFFHKQRA.HUF.) 
(PIUP)(PMD)  TCD  if1  MTBFi 


(l-R!Pi) (NRTSi) (DMHi)  + TE 


The  first  and  second  termrs  in  3-C  are  respectively 

training  costs  at  base  and  depot  level.  The  equations  first 

determine  life  cycle  manhour  requirements.  For  example, 

depot  life  cycle  manhour  requirement  = 

(TFFH)  (QPA.)  (UFi)  (1-RIP^  (NRTS^ ) (DMH  ) 

<MTBFi) 

Dividing  this  term  by  (PIUP)  times  (PMD)  gives  personnel 
required  at  depot.  Multiplying  this  by  ( 1+ (PIUP) (TRD) ) 
accounts  for  personnel  turnover.  All  of  this  is  then 
multiplied  by  TCD,  the  cost  of  training  one  man. 

Cost  Element  C,  is  excluded  from  both  TLSC-COD  and 

O 

TLSC-system,  because  it  is  considered  a negligible  input. 

In  those  applications  where  it  is  not  negligible,  the  same 
questions  of  relevance  as  those  discussed  in  relation  to 
equations  C^  end  C,  must  be  answered.  If  Air  Force 
policy  requires  the  number  of  personnel  assigned  to 
the  system  to  be  in  excess  of  actual  requirements  then  the 
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relevance  of  the  personnel  costs  is  not  totally  clear. 

7.  MANAGEMENT  AND  TECHNICAL  DATA 

Equation  seven  accounts  for  the  costs  of  management 
and  technical  data,  i.e., 

<307)  ■ cost  oi  management  and  technical  data 

n <gfA.  ) (UF  ) 

=*  1 — IMR0<-(1-RIP.)  (HRF+SR+TR)  JBLR 

-2  i«l  Fi 

TFK'i 

+ ~ ~ i.’-iKO+.l  (SR+TR)  ]BLR+TD(JJth) 

The  first  term  here  is  the  manhour  cost  of  completing 
forms  and  administrative  tasks  during  routine  main*er^nce. 

The  second  term  similarly  accounts  for  admin  is.  L: . tive 
costs  during  scheduled  maintenance.  The  last  term  represents 
the  purchase  cost  of  technical  data.  Cost  element  C7  is 
considered  negligible  and  is  omitted  from  TLSC-system  and 
TLSC-COD  computations. 

8.  FACILITIES 

Equation  eight  is  the  cost  of  facilities,  i.e., 

(3-8)  c0  = cost  of  facilities 
b 

= FD  + ( M)  (FB) 

These  terms  describe  the  costs  of  new  or  special 
facilities  for  base  or  depot  support.  These  costs  are 
relevant  to  the  decision  only  if  they *would  not  be  incurred 
if  the  system  were  not  procured  and  operated.  As  has  been 
pointed  out  by  Hitch  and  McKean  (32:138),  the  allocation 
of  this  entire  cost  to  the  weapons  system  in  question 
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impliticly  assumes  that  these  facilities  depreciate  ut  t;he 
sane  rate  as  the  weapons  system.  This  may  not  always  ise 
true,  and  yet  it  may  be  necessary  to  make  the  assumption 
simply  because  there  is  inadequate  supporting  data  for 
making  a more  accurate  assumption,  it  is  well  .o  note 
that  these  same  comments  could  be  apoiied  to  personnel 
training  as  well  as  facilities  since  this  training  pre- 
sumably has  some  finite  life  and  some  salvage  value. 

In  this  application  the  costs  of  new  facilities  are 
held  to  be  negligible,  and  equation  eight  is  omitted  from 
both  TLSC-COD  and  TLSC-system. 

9.  ENGINE  RELATED  COSTS 

Equation  nine  is  fuel  cost,  i.e., 

(3-9)  C9  * cost  of  fuel  consumption 
* (TFFH) (EPA) (FR) (FC) 

This  equation  calculates  the  life  cycle  fuel  costs.  It  is 
not  applicable  to  either  TLSC-COD  or  TLSC-system. 

10.  SPARE  ENGINES 

Equation  ten  is  the  cost  of  spare  engines,  i.e., 
(3-10)  C^q  * cost  of  spare  engines 
= ( (LS) (X)  + Y EUC 


where  X is  base  stock  and  Y is  depot  pipeline  spares, 


The 


value  of  X is  determined  by  the  mean  demand  rate  ' 

by  the  weighted  base  pipeline  time  (ERTS) (BP)  + (1-ERTS) (ARBUT) 
and  by  CONF,  the  policy  established  confidence  level  of 
availability.  X is  the  minimum  value  which  satisfies 
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E ^'ARGBH^GB)n  > CONF 
n-0  n‘ 

Here  A RGB  is  the  mean  of  a Poisson  distribution  of  demand. 
ARGB  is  the  product  of  demand  rate  and  base  pipeline  time. 

$o  ARGB  represents  the  mean  number  of  demands  per  pipeline 
cycle  time. 

Equation  ten  is,  of  course,  not  applicable  to  TLSOCOD 
or  TLSC-system. 

A summary  of  applicable  costs  is  presented  in  Table  IV. 


TABLE  IV 


APPLICABLE 

TLSC-COD 
- spares 

C2  - on  equipment 
maintenance 

C.  - off  equipment 
maintenance 


COSTS 

TLSC-SYSTEM 

C^  - spares 

C_  - on  equipment 
* maintenance 

C.  - off  equipment 
maintenance 

C5  - support  equipment 


This  chapter  has  discussed  the  important  details  of  the 
F-16  COD  incentive  contracting  provisions  as  well  as  the 
generalized  AFLC  Logistics  Support  Cost  Model.  Both  of  these 
subjects  are  prerequisites  to  a thorough  understanding  of  the 
analysis  which  is  to  follow.  The  presentation  of  the  AFLC 
LSC  model  should  facilitate  a generalization  of  the  total 
methodology  to  be  developed  here. 

In  the  next  chapter,  the  analytical  methodology  is 
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Chapter  4 
THE  ANALYSIS 


Introduction 

In  this  chapter , a complete  description  of  the  mathe- 
matical analysis  is  presented.  Those  concepts  which  would 
not  be  universally  understood  are  explained  in  an  expository 
fashion.  The  mathematical  details  which  are  not  essential 
to  an  understanding  of  the  material  are  omitted  from  this 
chapter  and  included  instead  in  Appendix  D where  the 
interested  reader  can  verify  the  correctness  of  the  conplete 
methodology. 

In  order  to  avoid  any  misunderstanding  or  inappro- 
priate extrapolation  of  the  results  of  this  study,  the 
fundamental  assumptions  upon  which  the  study  rests  will  be 
carefully  explained  at  the  onset.  Later,  the  effects  of 
relaxing  some  of  these  assumptions  are  examined.  It  is 
important  to  recall,  that  in  addition  to  the  assumptions 
listed  here,  the  study  is  also  dependent  upon  those  assump- 
tions already  described  upon  which  the  logistic  support 
cost  model  is  based. 

Use  of  Probability  Distributions 

Some  mention  has  already  been  made  of  the  probability 
distributions  which  will  be  used  here  in  an  attempt  to 
represent  reality.  Before  going  further  it  is  appropriate 
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to  more  carefully  describe  the  concept  of  probability  dis- 
tributions. If  a set  of  events  has  some  theorem ical  dis- 
tribution* as  depicted  in  Figure  7 below*  it  can  be  said, 
for  example,  that  about  34  per  cent  of  the  observ  tions 
sampled  from  that  population  will  rail  in  the  interval  AB. 
Or,  equivalently,  it  can  Lo  said  that  the  likelihood  of  any 
one  observation  falling  into  that  interval  is  .34.  Similar 
statements  can  be  made  regarding  the  probability  that  a 
single  observation  from  the  distribution  will  fall  into  any 
given  interval. 


Figure  7.  Probability  Distribution 

The  distributions  which  will  frequently  be  used  in  this  anal- 
ysis will  be  cumulative  distributions.  A cumulative  distri- 
bution function  (CDF)  will  depict,  for  any  given  value,  the 
fraction  of  observations  which  will  be  either  less  than  or 
equal  to  that  value.  In  Figure  8 below,  for  example,  the 
point  A is  that  value  below  which  about  16  per  cent  of  the 
observations  will  fall.  The  point  B is  that  point  below 
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which  SO  per  cent  of  the  observations  will  fall. 


Figure  8.  Cumulative  Probability  Distribution 

This  analysis  is  based  on  three  probability  distri- 
butions. Both  the  theory,  and  some  empirical  data  support 
the  assertion  that  these  distributions  are  a fair  repre- 
sentation of  the  real  world  distributions  which  they  are 
supposed  to  describe.  Of  course,  if  a real  world  distri- 
bution can  be  adequately  described  by  existing  data,  then 
that  real  world  (empirical)  pdf  ought  to  be  used.  As  stated 
earlier,  however,  insufficient  data  exists  to  construct 
empirical  distributions  for  any  of  the  activities  addressed 
in  the  analysis.  Hence,  the  actual  approach  used  here  has 
been  to  first  determine  what  distribution  the  theory  would 
suggest.  This  theory  is  then  checked  against  reality  by 
determining  what  distributions  provide  the  best  represen- 
tation of  applicable  past  experience.  Finally,  if  the  real 
world  data  suggests  some  uncertainty  as  to  the  correct 
representation,  then  a sensitivity  analysis  is  conducted 
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to'  determined  the  impact  of  an  incorrect  choice  of  distri- 
bution*. 

Time  to  Failure  Probability  Distribution 

Consider  first  the  probability  distribution  of  the  life- 
times of  each  FLU.  It  is  commonly  assumed  that  the  failure 
rate,  X,  for  electronic  equipment,  is  not  a function  of  the 
age  of  the  equipment.  That  is,  if  the  equipment  has  been 
operated  for  50  or  500  hours,  its  probability  of  failure  in 
the  next  instant  is  the  same  as  that  for  a piece  of  equip- 
ment which  has  operated  only  for  two  hours.  If  this  con- 
stant failure  rate  can  be  accepted  as  a fair  representation 
of  the  failure  characteristics  of  the  equipment,  then  it  can 
be  shown  mathematically  that  this  equipment  has  an  exponential 
distribution  of  life  times.1  Figure  9 depicts  the  exponential 
distribution. 


MEAN 

Figure  9.  The  Exponential  Distribution 


See  Appendix  D lot  proof. 
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In  order  to  check  this  assumption  of  exponential  lifetimes 
against  past  history,  the  records  of  Category  II  reliability 
and  maintainability  tests  have  been  consulted.  These  docu- 
ments provide  carefully  measured  empirical  data  which  has 
been  "fitted"  by  statistical  methods.  A total  of  eleven 
avionics  equipments  from  the  A7D  Category  II  tests  were 
checked  (49) . The  results  are  shown  in  Table  V below. 

TABLE  V 

DISTRIBUTION  OP  LIFETIMES 
Airborne  Electronic  Equipment 


A7D 

Total  avionics  equipments  11 

Distribution  Number 

Exponential  time  to  failure  € 

Weibull  time  to  failure  2 

Lognormal  time  to  failure  3 


Of  these  11  equipments,  four  are  the  actual  systems 
upon  which  the  F-16  control  FLU  reliability  predictions  are 
based.  Of  this  group  of  four,  two  are  exponential,  one 
Weibull  and  one  lognormal.  Of  course,  this  sample  is  not 
adequate  alone  to  provide  convincing  evidence  of  the 
correctness  of  the  exponential  time  to  failure.  The 
assumption  is  based  on  three  points  all  taken  together: 

1.  The  theory  strongly  supports  the  exponential 
assumption. 

2.  The  A7D  data  which  is  taken  from  similar  equipment 
operating  in  a fighter  environment  tends  to  support  th 
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thtory. 

3.  Sensitivity  analysis  on  the  uncertainty  of  LSC 

as  a function  of  MTBF  shows  that  an  assumption  of  Weibull 

failures  yields  results  which  are  not  significantly 

different  from  the  exponential  assumption  at  the  95  per 

2 

cent  confidence  level. 

Time  to  Repair  Probability  Distribution 

The  next  probability  distribution  to  be  considered  is 
the  distribution  of  time  to  repair.  As  was  stated  earlier, 
the  intention  here  was  to  find  one  type  of  distribution  to 
represent  maintenance  activities  on  the  flight  line,  at  base 
level,  and  at  depot.  The  literature  of  maintainability  is 
almost  unanimous  in  the  belief  that  this  distribution  is 
lognormal.  From  a purely  theoretical  standpoint,  there  is 
strong  support  for  the  lognormality . Behavioral  studies 
have  shown  that  when  human  beings  participate  in  a complex 
task  which  involves  classification  and  categorization,  the 
probability  distribution  of  tiroes  to  complete  the  tasx  is 
best  represented  by  the  lognormal  (26:54).  Since  avoinics 
trouble  shooting  and  repair  are  tasks  which  involve  the  use 
of  test  equipment  to  isolate  and  categorize  faults,  it  is 
not  surprising  that  these  tasks  would  have  the  lognormal 
distribution.  The  asymmetry  of  this  distribution  is 

_____________ 

For  this  test  a Weibull  distribution  with  shape  parameter  = 
1.1  was  used,  simulating  an  equipment  whose  failure  rate  would 
increase  slightly  over  time. 
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intuitively  appropriate  as  a representation  of  time  to  re- 
pair. It  would  be  expected  that  most  repair  times  would 
be  clustered  around  some  relatively  low  value,  with  a few 
observations  (for  example,  when  no  fault  is  located  ) quite 
far  out  to  the  right.  There  are  other  distributions  which 
possess  this  asymmetry,  but  none  which  derive  from  the 
categorisation  process  described  above  (26:55).  The  dis- 
tribution is  shown  in  Figure  10  below. 


Figure  10.  The  Lognormal  Distribution 


The  amount  of  skewness  is  controlled  by  the  variance  of 
the  distribution.  In  order  to  compare  the  theory  above 
with  real  world  observations,  the  Category  II  maintain-- 
ability  tests  were  reviewed  for  consistency.3  First,  con- 
sidering the  A7D  data  mentioned  above;  of  the  11  equipments, 
five  were  lognormal,  four  were  Weibull,  and  two  had  no  fit. 
These  data  alone  would  seer,  to  provide  a somewhat  weak 
confirmation  of  the  theory.  Of  those  four  equipments 


Vhese  data  were  taken  from  Air  Force  Flight  Test  Center 
Category  II  Reliability  and  Maintainability  Test  reports 
(49;  50;  51). 
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which  were  baseline  for  the  F-lt,  two  were  lognormal  and 
two  Weibull.  Since  other  maintainability  data  wore  avail- 
able from  recent  Category  II  tests  on  F-lll  and  C-5A 
avionics,  those  data  were  also  reviewed.  The  resells  for 
34  avionics  equipments  are  summarized  in  Table  VI  below. 

TABLE  VT 

MAINTAINABILITY  DATA 


Best  Fit  Number 

Lognormal  18 

Weibull  7 

Exponential  2 

No  Fit  __6 

Total  34 


Of  34  modern  avionics  equipments,  then,  53  per  cent  tnd  j 
lognormal  time  to  repair,  with  the  remainder  scattered 
between  the  Weibull,  the  exponential,  and  no  This 

is  strong  evidence  for  acceptance  of  the  lognormal  dis- 
tribution. 

Specifying  the  correct  lognormal  distribution  U~ 
somewhat  more  complicated  than  specifying  a cor. cot  exponen- 
tial distribution  because  the  lognormal  is  a two  parameter 
distribution  whereas  the  exponential  is  a ono  parameter 
distribution.  In  the  case  of  the  exponential,  the  stand- 
ard deviation  was  specified  at  the  time  as  the  mean,  because 
the  exponential  standard  deviation  is  equal  to  the  mean.  No 
particular  relationship  exists  between  the  mean  and  variance 
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| of  a lognormal  distribution.  To  solve  this  problem  a re- 

! view  of  the  standard  deviation  to  mean  ratio  of  the  18 

observed  lognormal  distributions  above  was  undertaken. 

It  was  determined  that  the  average  standard  deviation  to 
mean  ratio  was  1.2.  For  both  fighter  types  of  aircraft * 
<A7D  and  F-lll)  the  ratio  was  very  close  to  1.0  (.929  and 
.975  respectively) . After  considering  all  of  the  above 
information,  a standard  deviation  to  mean  ratio  of  1.0 
was  assigned  for  all  lognormal  maintenance  tasks. 

Probability  Distribution  for 
V Fraction  Reparable  this  Station 

r*  Determination  of  the  correct  distribution  to  repre- 

sent the  reparable  this  station  (RTS) /not  reparable  this 
station  (NRTS)  decision  is  a somewhat  different  matter. 
This  decision  is  what  is  often  called  a Bernoulli  trial 
in  which  one  of  two  outcomes  must  occur.  A toss  of  a 
biased  (or  unbiased)  coin  is  the  usual  example  of  a 
Bernoulli  trial.  This  process  is  represented  by  the 
binomial  distribution.  The  following  formal  assumptions 
are  necessary  to  satisfy  the  binomial  distribution. 

1.  The  experiment  consists  of  n identical  trials. 

2.  Each  trial  results  in  one  of  two  outcomes, 
usually  called  success  or  failure. 

3.  The  probability  of  success,  p,  is  constant  from 
trial  to  trial.  The  probability  of  failure  is  (l-p)-q. 

4.  The  trials  are  independent. 

If  RTS  is  labeled  as  i success  and  NATS  ap  a failure  (or 

i , . ■ . 


\ 


1 

! 
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vice  versa)  then  it  is  clear  that  this  process  will  satisfy 
the  assumptions  of  the  binomial  distribution,  it  is,  of 
course,  possible  that  the  value  of  p might  be  changed  over 
time  by  modifications  to  the  equipment,  but  it  see.as  safe 
to  assume  that  its  true  value  will  be  nearly  constant  over 
a 3500  hour  test. 


Selection  of  the  Analytical  Approach 

In  this  thesis,  the  simulation  approach  has  beoa  used 
to  develop  a relationship  between  the  stochastic  input  param- 
eters and  the  probability  distribution  of  MLSC's-  Tn  the 
following  discussion,  the  drawbacks  associated  with  the 
other  approaches  are  described. 

It  was  stated  earlier  that  there  are  several  methods 
which  could  be  used  tc  attack  this  problem.  Tn  fart,  one 
of  these  methods,  error  theory  approximation,  was  used  in 
initial  attempts  in  this  thesis.  It  was  found  in  these 
attempts  that  this  method  breaks  down  wh<-*n  attempting  *-<> 
deal  with  the  stock  level  calculations.  The  following 
derivation  will  serve  to  illustrate  some  of  the  dif  f ir;u!  iet> . 

Using  error  theory,  the  purpose  is  to  find  the  vari- 
ance of  MLSC  as  a function  of  the  variance  of  MTBF,  RTS,  and 


MMH. 


First,  it  is  true  that: 


V(MLSC)«V(C1)+V(C2) tV(C3)+V(C5) 

-V(Cu)tV(C21)+V(C3l)tV(C51>+.  . . 

.•■+V(C1,13)+V(C2,13)W(C3,13)*V,C5,13) 
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Considering  each  FLU  there  are  52  such  terms.  Almost  every 
one  of  these  terms  contains  MTBF,  MMH,  and  RTS.  For  each  of 
these  terms  in  which  there  is  a common  random  variable  which 
cannot  be  factored  out,  a covariance  term  develops.  There 
could  be  a very  large  number  of  these  covariance  terms. 

To  simplify  the  problem,  assume  RTS  and  HHH  are  not 
random  variables.  Now  flTBF  can  be  factored  from  each  of 
the  equations.  Considering  only  equation  for  simplicity: 

Cx  - (STK^  (M)  (UCi) 

1 N 

+ £ (PFFH)  (UFi)  (QPAi)  (l-RIPi)  (NRTSi) 

1*1 

• (DRCT)«(UCi) 

(After  deleting  the  second  term  since  COND»0  as  explained 
earlier.) 

And  dealing  with  one  FLU  for  simplicity: 

- V(STK*M*UC]  +vljpjfgpl  (PFFH'UF*  (1-RIP)  *NRTS 

• DRCT-UCJ2 

♦ COVKSTK.M.UC),  P.FFH • UC - ( 1-  W ; UC , 

This  co-variance  term  arises  since  STK  is  a function  of  MTBF. 

Now  looking  only  at  V(STK*M*UC)  and  recalling  the 
definition  of  STK:  STK  is  the  minimum  value  which  satisfies 

the  inequality 

I ( X-STK)  P (X j it) <EBO 
x>STK 

where:  It  - ****&$?'  * ( RTS » BRCT 

+NRTS (OSTCON ( 1-OS)  +OSTOS*OS) 
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The  above  equation  must  be  solved  explicitly,  foi  STK  before 
the  variance  of  STK  can  be  written  as  a function  ot  MT3F. 
Unfortunately  it  is  not  possible  to  solve  the  equation  lor 
STK  while  that  term  is  present  in  both  the  index  oi  the 
summation  and  the  summation  itself.  The  circumvention  of 
this  problem  would  by  no  means  establish  a cxeai  CO  an 

analytical  expression,  but  it  is  not  necesbary  to  consider 
explicitly  the  further  difficulties  expected  when  a solu- 
tion to  the  above  problem  is  not  apparent. 

In  further  support  of  the  simulation  approach  to  the 
problem,  the  following  observation  by  F.S.  Timson  in 
offered. 

The  problem  that  there  may  be  differences  ua,..u'. 
the  results  obtained  with  the  two  methods  (Monte  Carlo 
vs.  Error  Theory)  is  that  the  probability  distributions 
involved  may  not  be  normal.  Monte  Carlo  can  be  used 
with  probability  distributions  having  any  form  while 
propagation  of  error  requires  that  all  distributions 
be  normal;  hence,  in  situations  involving  non  normal 
distributions,  Honte  Carlo  calculations  with  large 
samples  should  yield  results  that  are  closer  to 
reality  (61:123). 

A final  observation  in  favor  of  the  simulation  approaev. 
is  that  it  can  accommodate  empirical  distributions.  This 
last  point  is  important  in  that  the  best  representation  of 
reality  would  be  derived  from  a simulation  which  used  actual 
empirical  distributions  rather  than  theoretical  approxima- 
tions. 


In  summary,  the  advantages  of  simulation  are:  1.  Abil- 

ity to  circumvent  mathematically  intractable  situations. 

2.  Greater  accuracy  when  the  underlying  distributions  are 
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non-normal.  3.  Greater  generality  of  application  since 
virtually  any  sort  of  input  distribution  can  be  used. 

Monte  Carlo  Simulation 

As  stated  in  the  last  section  a simulation  technique 
was  used  to  develop  an  output  distribution  of  MLSC's  as  a 
; /function  of  appropriate  random  inputs.  Zt  is  appropriate 
now  to  clarify  the  inputs*  the  output*  and  the  method  used 
to  develop  the  relationship.  Figure  11  below  is  a repro- 
duction of  Figure  3 from  Chapter  1 with  some  details  added. 


Tin*  to 
Failure 


Time  to 
Repair 


RTS 


INPUT  UNCERTAINTY 


LOGISTIC 

SUPPORT 

COST 

MODEL 


RUN  3500 
HOUR  TEST 
1000  TIMES 


HLSC  UNCERTAINTY 


DISTRIBUTION 
OF  MLSC's 


Figure  a.  Uncertainty  Sources 


The  first  question  which  arises  is  how  to  simulate  the 
input  uncertainty  which  is  Known  to  be  represented  by  the 
distributions  above.  The  first  step  is  to  convert  the 
continuous  PDF's  of  the  inputs  into  cumulative  distributions. 
Taking  the  lognormal  distribution  as  an  example*  the 


77 


OOR/t«/?SO-6 


cumulative  distribution  would  have  an  appearance  something 
like  that  shown  in  Figure  12. 


Figure  12.  Random  Sample  From  Lognormal  Distribution 


Next  it  is  necessary  to  generate  a random  number  input. 

Many  computer  routines  are  available  to  generate  rai.uom 
fractions  between  0 and  1.0.*  Given  a random  fraction  y, 
0<y<l,  the  point  x«P_1(y)  (where  F~ 1 ( y > is  the  inverse 
function  of  the  lognormal  CDF,  F(x))  represents  a random 
orservation  from  the  lognormal  distribution,  F(x)  . If  a 
large  number  of  these  samples  are  taken,  say  1,000,  and 
plotted  in  a histogram,  that  histogram  would  resemble  veiy 
closely  the  lognormal  distribution  above.  This  then  as 
the  general  method  used  to  generate  samples  from  a given 
continuous  distribution. 

The  generation  of  random  variables  from  th«  binomial 
distribution  is  somewhat  simpler.  Here  it  is  only  necessary 
to  generate  a random  fraction  and  check  to  determine  whether 


1 


See  Appendix  D for  discussion  of  random  number  generation. 
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it  is  above  or  below  a given  value.  For  example,  assume 
an  RTS  value  of  .6.  This  process  could  be  simulated  by 
generating  random  fractions,  and  for  all  values  .6, 
designate  those  observations  reparable  this  station.  All 
observations  greater  than  .6  will  be  designated  not 
reparable  this  station. 

The  mathematical  device  used  to  convert  a random 
fraction  to  a random  observation  on  a particular  CFD  is 
called  a process  generator.  For  the  binomial  distri- 
bution, the  process  generator  is  simply  the  random  number 
generator  itself.  For  the  continuous  distribution  used 
here,  the  process  generators  are  more  complex.  The  deri- 
vations for  these  process  generators  are  shown  in  Appendix 
D.  It  is  sufficient  here  to  simply  list  the  generators  to 
be  used.  They  are  listed  in  Table  VII  on  the  next  page. 


79 


lOOR/SM/75D-6 


TABLE  VIZ 

PROCESS  GENERATORS 

DISTRIBUTION  GENERATOR 

Exponential  Time  to  Failure*MTBP (Ln (RAND) ) 

Lognormal  V*  ( (-2Ln  (RAND1) ) ' 5)co»( 2*RAND2) 


V1=V • SD  + U 
Manhours-2 . 7 18^ 

Weibull  Time  To  Fai luregMTBP  (Ln(RAND) ) 1/e2 

r(I+i/»2) 

Manhour s»f Contract)  . 

\ Mean  > * (Ln(RAND))  ' 2 

LManhoursJ 

r n+r/e2) 

'^Binomial  The  Random  Number  Generator 

RAND  ia  used  to  produce  binomia 1 ly 
distributed  variates  to  stimulate 
RTS 


Note  1:  RAND  is  a random  number  generator  producing  random 

numbers  uniform  on  the  interval  (0,1) 

Note  2:  RANDl  and  RAND2  are  independent  random  numbers, 

uniform  on  the  interval  (0,1) 

Note  3:  v and  VI  are  dummy  parameters  for  the  lognormal 

generator,  w and  SD  are  the  mean  and  standard 
deviation  of  the  lognormal  distribution. 

Note  4t  ©2  is  the  shape  parameter  of  the  Weibull  distribu- 
tion. 


Given  a method  for  obtaining  stochastic  time  to  failure, 
manhours,  and  RTS  the  next  step  is  to  simulate  the  sampling 
of  these  values  so  that  they  would  appear  just  as  though 
they  had  been  observations  from  a 3,500  flying  hour  test 
program. 


I ) 


l 


A 
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From  this  point,  to  follow  the  logic  of  the  computer 
model  which  was  used  it  will  be  necessary  to  refer  to 
Figure  13,  the  abbreviated  computer  flowchart.  The  complete 
listing  of  the  program  is  included  in  Appendix  D for  the 
interested  reader.  In  blocks  1 and  2 the  computer  reads  all 
inputs  including  FLU  data  such  as  RMH,  RIP,  RTS,  and  user 
inputs  such  as  choice  of  probability  functions  and  stochastic 
variables.  Assume  here  that  the  program  chosen  is  for  all 
‘three  variables  stochastically  determined  with  exponential 
time  to  failure  and  lognormal  maintenance  activity.  Given 
these  inputs,  the  program  flow  is  to  Block  9 where  expo- 
nential times  to  failure  are  generated.  These  lifetimes  are 
recursively  generated  and  added  together  until  their  sum  is 
greater  than  3,500.  Then  to  find  measured  MTBF  (TBFM) , 

3,500  is  divided  by  the  number  of  failures  minus  one  (I-l) . 
The  rationale  for  this  procedure  is  that  in  an  actual  3,500 
hour  test  there  would  probably  be  no  failure  at  precisely 
the  3,500  hour  point;  rather,  there  would  i-l  failures 
before  the  3,500  hour  point;  and  another  (the  ith)  some- 
time after  the  3,500  hour  point.  The  actual  procedure  in 
the  real  test  would  be  to  divide  3,500  by  the  number  of 
failures  which  had  occurred  up  to  that  point  or  I-l.  Of 
course  in  the  simulation,  failures  must  be  generated  up 
through  the  ith  which  is  the  first  one  to  occur  after 
3,500  hours  elapsed.  There  is  no  other  way  of  determining 
I-l.  If  a Weibull  failure  rate  were  chosen,  the  determina- 
tion of  TBFH  would  proceed  in  an  exactly  parallel  fashion. 
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Figure  13.  (continued) 
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Or  if  non-stochastic  MTBF's  are  desired,  then  TBFM  is  set 
equal  to  TBF.  (TBF  is  the  computer  variable  name  fo.  con- 
tract specified  MTBF.) 

In  the  next  step.  Block  13,  the  measured  RTS  called 
RTSM  is  determined.  Every  failure  which  occurred  during 
the  test  is  subjected  to  the  binomial  sampling  procedure, 
to  determine  whether  it  will  be  repaired  at  this  station  or 
sent  to  depot.  After  all  (1-1)  failures  have  been  disposed 
of,  then  RTSM  is  just  the  number  repaired  on  stat.on 
divided  by  (1-1) . Symbolically,  RTSM= I BASE/ ( I - 1 ) . Since 
base  level  condemnations  are  ^ero  here  NRTSM  is  i ust  1-RTSMj 
and  the  number  repaired  at  depot  is  just  (I-l)-JBASF. 

Once  the  numbers  repaired  at  base  and  depot  l«.vel  are 
determined,  then  appropriate  samples  may  be  taken  from  each 
of  the  four  manhour  distributions.  Block  18.  For  PAMH, 
there  are  (1-1)  samples,  s’nce  every  failure  generates 
preparation  and  access  activity.  For  RMH  and  BMH  there  are 
IBASE  samples  taken.  For  DMH  there  are  IDEPOT  samples 
taken.  Finally  then  in  Block  22  the  average  values  for 
each  of  these  variables  over  the  3,500  hour  test  are  found. 

In  Block  23,  the  stock  level  calculation  is  performed 
based  on  the  measured  values  of  TBFM  and  RTSM.  When  the 
stock  level  (STK)  is  known,  then  the  program  can  compute 
the  /alues  of  each  of  the  cost  equations,  , C and  C^» 
based  on  the  measured  inputs  for  MTBF,  MMH , RTS  and  STK. 

The  sum  of  C^+Cj+C^  then  is  the  measured  logistic  support 
cost  (MLSC)  for  one  FLU. 
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The  same  identical  sequence  then  must  be  run  for  each 
of  the  13  FLUs,  each  time  adding  an  increment  to  the  value 
of  MLSC . When  all  FLUs  are  included,  then  the  result  is 
one  sample  point  of  MLSC. 

Determining  the  Form  of 
the  MLSC  Distribution 

With  this  group  of  1,000  observations  of  MLSC  an 
empirical  probability  distribution  of  MLSC's  may  be  con- 
structed. To  assist  in  this  task,  the  computer  model  above 
has  a statistical  routine.  This  routine  sorts  all  of  the 
observed  MLSC  values  into  ascending  order,  calculates  a f 

j 

frequency  distribution  based  on  20  equal  intervals,  and  I: 

j 1 

prints  a histogram  of  observations.  Figure  14  below  shows 
a sample  of  a typical  histogram  printout.  Also  included  in 
the  output  are  the  mean  and  standard  deviation  of  the  sample  i 

ot  1,000  MLSC's.  With  the  above  information  available  it 
is  possible  to  determine  the  form  of  the  probability  dis- 
tribution. The  first  step  is  to  form  a null  hypothesis, 
as  to  the  true  form  of  the  distribution.  The  histogram 
is  very  useful  in  this  step,  and  from  observation  of  Figure 
14,  it  would  be  natural  to  consider  the  hypothesis  that  the  i 

dsitribution  is  normal.  It  may  be  of  some  benefit  to  plot  j 

the  cumulative  distribution  on  normal  probability  paper; 
here  a straight  line  is  indicative  of  normality.  The 
distribution  below  is  shown  on  normal  probability  paper 
in  Figure  15.  Clearly  the  distribution  exhibits  the  char- 
acteristics of  normality  except  at  the  extremes.  At 

! 
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Figure  14.  Typical  Output  Histogram  of  MLSC's 
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Figure  15.  Distribution  of  MLSC's  on  Normal  Probability 
Paper  (Center  section  has  been  deleted) . 
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At  approximately  i,j  standard  deviations  from  the  mean  there 
is  some  truncation  shown  by  the  curved  lines  at  the  top  and 
bottom  in  Figure  12.  The  tact  that  some  truncation  occurs 
is  not  particularly  surprising  in  a sample  of  1,0CJ.  A 
sample  of  5,000  should  eliminate  this. 

Having  formed  the  null  hypothesis  that  the  distribution 
is  normal,  it  is  next  necessary  to  check  this  assumption 
statistically.  The  method  used  here  is  the  Kolmogorov- 
Smirnov  (K-S)  goodness  of  fit  test.  In  this  test,  the 
cumulative  distribution  function  of  the  empirical  data  is 
compared  with  the  cumulative  distribution  function  of  the 

1 

hypothesized  distribution.  If  at  any  point  in  the  compari- 

j 

son,  the  difference  between  the  two  functions  is  greater 
than  "D,"  the  K-S  statistic, the  null  hypothesis  must  be 

! 

rejected.  The  value  of  D is  determined  by  the  number  of  1 ! 

observations  in  the  sample  and  a,  the  desired  level  of 

risk  of  rejecting  a null  hypothesis  which  is  true.  In 

this  study  an  a of  .10  and  a sample  size  of  1,000  were 

used  in  all  testa.  This  gives  a D statistic  of  .0386. 

Since  the  K-S  test  is  more  intuitively  clear  in  a graphi- 
cal presentation,  the  K-S  goodness  of  fit  test  for  1,000  j 

samples  of  MLSC,  with  MTBF  as  a random  input  is  shown  : 

in  Figure  16.  Any  deviation  of  more  than  .0386  between  j 

the  two  depicted  lines,  is  cause  for  rejection  of  the 
null  hypothesis.  In  this  case,  the  null  hypothesis  is  not 
rejected.  In  fact,  the  largest  deviation  from  the  theoreti- 
cal distribution  is  only  .025  which  occurs  at  MLSC  ■ 41.66. 
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Interpretation  ot  tno  Simulation  Results 

Given  that  the  normality  hypothesis  ie  pt«d,  the 

probability  distribution  o.  HLSC's  can  now  be  coftipie  ely 
specified.  It  all  input  parameters  are  truly  eqna:  to 
their  contractual  taigecs,  uhu  distribution  of  HLiC's 

will  be  a normal  distributicn  with  niean  $38.4  mil  lion  and 
standard  deviation  $1.8  million.  That  distribution  is 
depicted  in  Figure  17. 


38.4  44  48 

MLSC  Millions 


Figure  17.  Normal  Distribution  of  MLSC's 

Sensitivity  Analysis  of  the 
Input  Distributions 

As  stated  earlier,  substitution  of  a Wei bull  time  to 
failure  distribution,  with  shape  parameter  1.1,  has 
virtually  no  effect  on  the  output  distribution  of  MLSC’s. 
Substitution  of  a Weibull  time  to  repair  distribution  tor 
the  lognormal  time  to  repair  distribution,  however,  has  a 
significant  effect  on  the  output  distribution  of  MLSC's. 

The  model  was  run  with  a Weibull  time  to  repair  distribution. 
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with  shape  parameter  of  . 18.  S Under  these  conditions*  the 
standard  deviation  of  the  MLSC  is  about  $12  million.  This 
is  about  three  times  the  average  value  of  $4  million  which 
occurs  when  the  lognormal  time  to  repair  distribution  is 
used.  (With  all  three  variables  stochastic  in  both  cases). 
It  is  apparent  that  the  small  shape  parameter  .18  is 
causing  a very  large  variance  in  the  time  to  repair  dis- 
tribution, since  the  standard  deviation  of  a Welbull  dis- 
tribution is: 

"wetbull  ■ /et  r<2/e2+l)-[E<x))2 

For  E(x)  » 1.0  the  standard  deviation  ist 

°Weibull  * 22,9 

This  is  a very  large  standard  deviation  indeed  for  a mean 
of  1.0. 

The  introduction  of  the  above  Weibull  distribution 
causes  not  only  a change  in  the  variance  of  the  MLSC  dis- 
tribution, but  a change  in  the  form  of  the  distribution 
as  well.  The  null  hypothesis  which  was  actually  formed 
in  this  thesis  was  that  the  distribution  was  Weibull  with 
scale  parameter  43.1  and  shape  parameter  2.8.  These 

^The  shape  parameter  of  .18  is  the  average  value  over 
the  seven  observations  of  the  equipments  which  demon- 
strated Weibull  times  to  repair  in  the  Category  II 
Reliability  and  Maintainability  Tests  cited  earlier 
in  this  thesis  (49;  SO;  51).  The  actual  values  «■  the 
seven  observations  were  (.31,  .08,  .09,  .13,  .1  .35, 

.15)  . 
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p«raMt«rf  wirt  estimated  using  the  method  of  moments  and 
special  techniques  described  in  Johnson  and  Kotz  (34). 

A Kolmogorov  Smirnov  (K-S)  test  conducted  at  the  90 
per  cent  confidence  level  causes  a rejection  of  this  null 
hypothesis.6  Having  rejected  the  hypothesised  distribution, 
it  can  only  be  said  that  the  distribution  has  a Weibull-like 
appearance  and  very  clearly  is  not  normal. 

It  is  certainly  evident  then  that  the  validity  of  the 
model  is  dependent  upon  a correct  selection  of  the  time  to 
repair  distribution.  Nonetheless,  the  choice  of  the  log- 
normal distribution  in  this  application  is  sufficiently 
well  founded  in  theory  and  supported  by  the  evidence,  that 
there  can  be  a reasonable  level  of  confidence  in  the  findings. 

Making  Statistical  Inferences 
Regarding  the  MLSC  Distribution 

Returning  to  the  consideration  of  the  normal  distri- 
bution of  MLSC's  which  results  from  inputs  of  exponential 
time  to  failure,  lognormal  time  to  repair,  and  binomial 
fraction  reparable  this  station;  it  is  now  possible  to  make 
some  statistical  inferences  regarding  this  distribution. 

For  example,  after  consulting  a table  of  the  normal  dis- 
tribution, it  is  possible  to  say  that  the  likelihood  of 
any  single  random  observation  from  the  above  distribution 
being  greater  than  $48  million  or  1.25  TLSC  is  .0003;  or. 


See  Appendix  D for  The  Kolmogorov  Smirnov  test. 
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in  10,000  samples,  about  three  observations  would  be  likely 
to  exceed  that  value.  (The  value  .0003  is  for  only  time  to 
failure  stochastic;  with  all  three  variables  stochastic 
ebout  20  observations  in  1000  would  exceed  this  value.) 

Now  suppose  all  specifications  are  not  met.  In 
particular,  suppose  that  all  of  the  true  MTOF's  are  equal 
to  .9  times  their  specified  value  and  that  all  NMH  and  RTS 
values  are  exactly  as  specified  in  the  contract.7  This 
would  cause  two  things  to  happen  to  the  distribution  of 
Figure  17.  Its  mean  would  move  to  the  right,  and  its 
standard  deviation  would  increase  somewhat.  If  the  NTBF's 
were  now  multiplied  by  .8,  the  mean  would  move  farther  to 
the  right,  and  the  standard  deviation  would  again  increase. 
Multiplying  the  MTBF's  by  various  factors  between  1.1  and 
.6  would  then  give  a series  of  probability  distributions 
along  the  horizontal  axis  as  depicted  in  Figure  18.  Each 
of  these  distributions  is  a normal  distribution,  but  each 
has  a different  standard  deviation.  If  the  relationship 
between  the  mean  and  standard  deviation  of  these  distri- 
butions could  be  determined,  then  it  would  be  possible  to 
construct  a distribution  at  virtually  any  point  along  the 
axis  without  repeating  the  simulation  procedure. 

7 The  computer  model  provides  a value  of  FACTOR  as  a 
multiplier  for  input  true  MT3F. 
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Figure  18.  Series  of  Normal  Distributions 

In  order  to  determine  this  relationship,  the  simulation 

model  was  run  for  14  values  of  FACTOR  between  1.1  and  .6. 

A regression  analysis  was  then  conducted  and  it  was 

determined  that  the  estimated  standard  deviation  (S) 

for  any  value  of  factor  could  be  found  by  the  following 
8 

equation  : 

S * 3.98  - 1.17  • FACTOR 

(33)  (8.17)  (t  statistic) 

This  equation  predicts  the  values  quite  well  for  the 

range  of  interest.  An  alternative  method  for  predicting 

the  standard  deviation  for  any  given  point  is  to  assume 

that  S is  a constant  percentage  of  the  mean.  This  method 

s 

The  regression  methodology  is  outside  the  scope  of  this 
thesis.  See  (48<37S)  fer  a discussion  of  this  topic. 
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is  acceptable  over  a very  short  interval,  say  from  factor  - 
1.1  to  factor  * .95. 9 

Given  that  a distribution  of  MLSC's  about  any  given 
mean  MLSC  can  be  established,  then  it  is  possible  to  make 
inferences  about  the  results  of  a test  conducted  under  any 
specified  value  of  factor.  For  example,  assume  the  true 
value  of  ail  MTBF's  is  equal  to  .75  times  the  specified 
value.  If  1,000  tests  of  3500  flying  hours  each  were  con- 
ducted under  those  circumstances,  the  distribution  of  MLSC's 
depicted  below  would  result. 


I 


MLSC  (million  of  dollars) 

Figure  19.  Normal  Distribution  Mean  $49.32 
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9 

In  order  to  compare  the  two  methods,  predictions  of  13 
values  of  s were  made  with  each  method.  These  predictions 
were  then  compared  with  the  known  values.  The  regression 
predictions  were  very  good  over  a range  of  FACTOR  from 
.6  to  1.1,  with  the  greatest  error  of  prediction  being 
about  5%  of  the  known  value.  Errors  using  the  percentage 
of  the  mean  method  over  this  range  were  as  much  as  30%  of 
the  known  values.  Over  the  range  of  FACTOR  from  .95  to 
1.1,  the  percentage  of  the  mean  predictions  were  off  by  no 
more  than  5%. 
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It  can  be  seen  then,  that  for  factor  = .75  or  equivalantly , 
true  mean  LSC  * $49.32  million,  about  663  observations  out 
of  1,000  would  be  expected  to  exceed  $48  million. ^ It 
would  also  be  true  that  for  a single  observation,  tne 
likelihood  of  exceeding  48  million  is  .663.  Under  these 
circumstances,  only  about  two  observations  in  10,000  would 
be  expected  to  be  less  than  38.4  million  or  the  probability 
of  a single  observation  being  under  38.4  million  is  .0002. 
Translating  these  statistics  into  contractual  terms;  with 
a true  LSC  of  49.32  million,  if  1,000  tests  of  3,500  hours 
were  run,  the  COD  clause  would  be  invoked  about  663  times. 
An  award  fee  would  be  presented  less  than  once.  Stated 
differently,  for  a single  observation  the  probability  of 
invoking  COD  would  be  .663. 

Similar  statistics  can  now  be  generated  for  any  true 
mean  value  of  LSC.  If  these  statistics  are  generated  and 
plotted  with  LSC  on  the  abscissa  and  probability  on  the 
ordinate,  a decision  curve  such  as  the  one  depicted  in 
Figure  20  results. 


It  should  be  noted  at  thin  point  that  the  LSC  itsvlf  is 
in  reality  a random  variable.  That  is,  if  it  were  possible 
to  measure  the  actual  15  year  support  cost  over  several 
replications,  with  all  exogenous  and  endogenous  variables 
equal,  then  the  results  would  likely  be  several  different 
observations  of  LSC.  This  is  consistent  with  the  definition 
of  a random  variable,  i.e.,  it  is  a numerical  event  whose 
value  will  vary  in  repeated  samplings.  Thus  there  can  be 
no  "true"  value  of  LSC,  but  only  a true  mean  value  of  LSC. 

In  the  remainder  of  this  thesis,  however,  the  term  true 
LSC  will  be  used  in  place  of  the  term,  true  mean  LSC,  for 
the  sake  of  simplicity  of  exposition. 
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The  two  curves  on  this  chart  were  derived  from  the 
two  statistics  just  discussed.  The  curve  on  the  left 
represents  the  probability  that  an  award  will  be  made  for 
any  given  true  value  of  LSC.  More  formally  stated,  the 
curve  is  the  conditional  probability  of  an  award,  given 
a value  of  true  LSC.  The  curve  on  the  right  is  simply 
the  probability  that  a correction  of  deficiencies  will  be 
invoked  given  a valu^  of  true  LSC. 

To  take  an  example,  suppose  the  true  value  of  LSC  is 
actually  43.6  million  where  the  letter  A*  appears  on  Figure 
20.  From  this  point  a vertical  line  to  intercept  the  award 
fee  curve  shows  that  when  this  is  the  true  value  of  LSC,  the 
probability  of  an  award  is  .05.  Intercepting  the  right  hand 
curve  shows  the  probability  of  invoking  COD  is  .17.  An 
equivalent  statement  would  be  that,  if  1,000  tests  of 
3,500  hours  each  were  conducted,  in  about  five  of  these 
tests  an  award  would  be  made  under  the  current  Support  Cost 
Guarantee  provision  and  in  about  seven  tests  the  COD  pro- 
visions would  be  invoked. 

Consider  now  only  the  right  hand  curve  from  Figure  20. 
This  curve,  which  has  a useful  interpretation  in  the  frame- 
work of  classical  statistics,  is  shown  alone  in  Figure  21. 

In  classical  statistical  hypothesis  testing,  this  co^ve 
represents  the  power  of  the  test.  In  particular,  the  curve 
of  Figure  21  is  the  power  curve  with  respect  to  the  null 
hypothesis. 

Ho:  u < TLSC, 
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Figure  21.  Fower  of  Test  Curve 
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where  g - mean  MLSC,  versus  the  alternative  itJ  pothusi  . 

Ha:  g > TLSC. 


Proceeding  from  the  assumption  of  normality , the 
power  curve  fun~Lion,  r(p)#  can  be  derived  analytically 
as  follows: 


Y { g ) = Power  (g)  * 1-b  (u> 


Y (g) 


* Pr  (Accept  Ha | g) 

= Pr  (MLSC  >1.25  TLSC|u) 

= pr  (MLSC-g  > 1,25  TLSC-g} 
9 (g)  8 ( g) 

= Pr  (Z  > l-aS.-TLSC^  , 

8 (g) 


cr 


twhere  8 (g)  is  the  sample  standard  deviation  as  a function 
of  g . 


In  classical  hypothesis  testing,  it  is  the  usual 
practice  to  specify  the  desired  levels  for  a,  known  as  the 
significant  of  the  test,  and  8 (g  ) , where  g is  some 
■Specified  value  of  g.  Given  these  values,  the  left  end 
point  of  the  critical  region  (1.25)  TLSC  in  this  case)  and 
the  sample  size  (test  length  in  this  case)  are  automatically 
determined.  But  in  this  application,  the  procedure  was 
reversed,  first  specifying  test  length  and  critical  region, 
and  then  accepting  the  rasulting  values  of  a and  6 (va) . 

Given  the  knowledge  of  the  variance  of  the  MLSC 
distribution  which  can  be  estimated  by  the  methods  described 
in  this  thesis,  a more  scientific  approach  to  the  statistical 
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testing  of  LCC  can  be  visualized.  First,  it  is  necessary 

to  develop  a general  expression  for  uhe  variance  of  MLSC 

in  terms  of  the  inputs  to  Cl,  C2  and  C3.  Next,  specify 

a,  u and  6 (u  ) in  the  contract.  These  values  would 
a a 

then  determine  the  required  test  length  and  the  left 
endpoint  of  the  critical  region. 

It  is  interesting  to  note  that  the  value  of  a which 
results  from  the  LSC  test  described  herein  is  v<»ry  small, 
i.e.  <_  .01. 

The  Experimental  Design 

All  of  the  curves  shown  so  far  have  been  those 
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derived  from  the  LsC  model  with  MTBF  as  a stochastic  input. 
Now  it  is  appropriate  to  investigate  the  effect  on  the 
model  of  assuming  that  maintenance  manhours  (MMH)  and 
reparable  this  station  (RTS)  are  rppl<iced  by  random  vari- 
ables. The  purpose  of  this  part  cf  the  investigation  is 
to  determine  which  variables  will  have  a statistically 
significant  effect  on  the  variance  of  the  MLSC.  If  it  can 
be  positively  determined  that  some  variable  has  no  signifi- 
cant effect  on  the  variance  of  the  MLSC,  then  it  would 
probably  be  reasonable  to  treat  that  variable  as  a deter- 
ministic input  in  this  model.  Note  here  that  it  would  have 
to  be  shown  not  only  that  the  variable  itself  does  not  have 
any  effect  but  also  that  it  has  no  significant  interactions 
with  any  of  the  other  variables.  If  such  a variable  is 
found,  this  would  mean  to  the  designer  of  a verification 
test  that  this  particular  variable  may  be  added  to  or 
deleted  from  the  model  with  no  significant  effect,  on  the 
amount  of  uncertainty  in  the  MLSC  distribution. 

The  second  purpose  of  this  part  of  the  investigation 
is  to  determine  the  relative  importance  of  each  of  the 
random  input  variables  in  terms  of  their  contribution  to 
the  overall  uncertainty  in  the  distribution  of  MLSC's. 

The  factor,  test  length  is  included  in  the  experimental 
design  for  the  same  reason  as  the  other  variables,  namely 
to  measure  its  contribution  to  the  uncertainty  in  the  dis- 
tribution of  MLSC's.  Of  course,  it  would  be  expected  that 
the  test  length  would  have  a negative  contribution  to  the 
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uncertainty.  That  is,  the  variance  of  the  MLSC  distri- 
bution ought  to  decrease  as  test  length  increases. 

This  investigation  has  been  carried  out  using  the 
techniques  of  experimental  design  and  analysis  of  variance. 
A discussion  of  the  methodology  involved  in  that  analysis 
is  outside  the  scope  of  this  thesis.  The  reader  is 
referred  to  references  (15;  42;  47}  for  discussions  of 
the  techniques. 

This  investigation  was  conducted  using  the  analytical 

4 

framework  of  a 2 complete  factorial  experiment;  meaning 
there  were  four  separate  factors  with  two  levels  each. 

The  four  factors  are:  MTBF,  MMH , RTS,  and  verification 

test  length.  The  levels  for  the  first  three  are  either 
deterministic  or  stochastic.  For  test  length  the  two 
levels  are  3,500  hours  and  10,000  hours.  The  response 
which  is  being  measured  as  a function  of  all  of  these 
inputs  is  the  uncertainty  (standard  deviation)  of  the 
MLSC.  Each  of  these  factors  could  be  examined  separately, 
but  this  would  not  be  a true  depiction  of  the  behavior  of 
the  system,  as  the  interaction  among  the  factors  must  be 
considered  as  well.  The  analysis  of  variance  techniques 
account  for  both  the  separate  and  the  interacting  effects 
of  each  factor.  Tables  VIIIA  and  VIIIB  depict  the 
experimental  design  and  the  results  of  the  analysis  of 
variance  in  the  traditional  format. 
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TABt£  VI I LA 

THE  EXPERIMENTAL  DESIGN 


Analysis  of  variance  Observations  Tib  Replications 
Hughes  Radar 


FACTCR  HTBF 


Deterministic 


Stochastic 


FACTOR  B,  HMH 

Deterministic  I Stochastic  I Deterministic 


1.08  I 16 
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Table  VIIZA  shows  the  observations  of  variance  of 
MLSC  from  two  full  replications  of  the  experiment.  The 
two  replications  were  identical  in  all  respects  except  for 
the  random  number  seed.  Table  VXIIB  shows  the  results  of 
the  analysis  which  was  conducted  with  a standard  IBM  360 
library  routine.  In  particular  the  "F"  statistic  in 
column  5 of  Table  VII IB  indicates  whether  or  not  the 
particular  effect  is  significant  at  the  90  per  oent  con- 
ficence  level.  In  this  experiment,  a factor  is  significant 
if  it  has  an  F statistic  > 4.5.  A total  of  eight  effects 
and  interactions  can  be  said  to  be  statistically  signifi- 
cant. All  individual  factors  are  significant.  The  inter- 
actions of  AC,  BD,  CD  and  ABC  are  significant.  The  meaning 


* ."4 


of  an  interaction  between  two  effects,  for  example,  A and  C 

is  that  the  response  given  with  C at  its  high  level  is  not 

independent  of  the  level  of  A.  Or  the  amount  of  uncertainty 

produced  by  the  random  variable  RTS,  depends  upon  whether 

MTBF  is  deterministic  or  stochastic.  The  other  interactions 

have  similar  explanations.  Figure  22  graphically  depicts  the 

interaction  between  A and  C.  Clearly,  the  amount  of  change 
Uncertainty 

of  MLSC  MTBF  Stochastic_ 

3. Of  ' 


Deterministic  Stochastic  Level  of 

Figure  22.  Interaction  Between  MTBF  and  RTS.  Factor 

C(RTS) 
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in  the  uncertainty  with  changer  in  RTS  depanda  o>,  the  level 
of  MTBF • With  HTBF  deterministic  a relatively  large  change 
"X"  occurs.  A smaller  change  "Y"  occurs  when  MTBF  is 
stochastic. 

With  the  significance  of  the  factors  and  interactions 
determined,  the  next  step  in  the  analysis  is  to  determine 
statistically  the  relative  importance  of  each  or  the  contri- 
butions to  the  uncertainty.  This  can  be  accomplished  by  r 
technique  known  as  contrasts  of  factor  level  means  whioh  is 
discussed  in  reference  (15) . This  is  a comparison  of  average 
response  averaged  over  each  of  the  input  random  variables. 

The  results  of  this  comparison  are  shown  in  Table  IX. 


TABLE  XX. 

Factor  Level  Means 


Analysis  of  Standard 

Deviation  of  mlsc 

Factor 

Mean  Response 

MTBF  Stochastic 

2.749 

| 

RTS  Stochastic 

2.839  ! 

EEgBBBBKM 

2.213 

1 

90  % Confidence  Intervale  j 

For  Difference 

Between  Means 

MTBF  - RTS 

-.07,  .25 

MTBF  - MMH 

.375,  .536 

.466,  .787 

From  the  tabled  information  above,  the  following  deductions 
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can  b«  made  with  a joint  confidence  of  90  per  cent! 

1.  There  is  no  significant  difference  between  the 
aean  response  with  MTBF  stochastic  and  that  with  RTS 
stochastic. 

2.  The  mean  response  with  MTBF  stochastic  is  greater 
than  that  with  MMH  stochastic  by  .4S5S  t .0805. 

3.  The  mean  response  with  RTS  stochastic  is  greater 
than  that  with  MMH  stochastic  by  .547  4 .0805. 

In  summary,  it  can  be  said  that  the  greatest  contri- 
butor to  the  uncertainty  of  MLSC  is  either  MTBF  or  RTSj 
there  is  no  significant  difference  in  their  contribution. 
MMH  contributes  somewhat  less  (about  20  per  cent  less)  to 
the  uncertainty  than  either  MTBF  or  RTS. 

This  information  is  useful  in  understanding  the 
importance  of  correct  measurements.  It  is  no  surprise  that 
MTBF  is  a major  contributor  to  uncertainty  of  MLSC  but  the 
fact  that  RTS  is  equally  important  may  not  have  been 
intuitively  apparent.  It  is  obvious  then  that  measurement 
errors  in  MTBF  and  RTS  will  contribute  significantly  to  the 
overall  uncertainty.  And  while  measurement  errors  of  MMH 
are  secondary  to  MTBF  and  RTS  they  are  only  about  20  per 
cent  less  important.  This  information  is  also  useful 
in  planning  a simulation  model  for  a given  verification 
test.  It  should  be  apparent  now  that  none  of  the  three 
factors  is  a negligible  contributor  to  uncertainty,  and 
that  ali  three  ought  to  be  included  if  they  are  all  subject 
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to  verification.  If  one  factor  must  be  omitt«.  5 . then  the 
omission  of  MMH  would  have  the  smallost  impact  on  th-* 
validity  of  the  model. 

This  discussion  of  analysis  of  variance  concludes  the 
presentation  of  the  formal  methodology. 

In  the  next  chapter,  several  applications  of  this 
simulation  model  will  be  explored  in  detail. 
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Chapter  5 
APPLICATIONS 

Introduction 

In  this  chaptc.  some  applications  of  the  simulation 
methodology  are  presented.  Three  applications  will  be 
developed: 

1.  Analysis  of  COD  invocation  ratio  and  verifi- 
cation test  length. 

2.  Analysis  of  a contractor  strategy. 

3.  An  award  fee  design. 

Application  to  Determinatior 
of  Suitable  Invocation  Ratio 
and  Verification  Test  Length 

The  first  application  to  be  addressed  is  determination 
of  a suitable  COD  invocation  ratio  and  verification  test 
length  for  a given  contract.  The  first  step  in  such  an 
analysis  is  to  establish  a conditional  distribution  of 
MLSC's  at  any  point  along  the  MLSC  continuum.  This  is  most 
easily  accomplished  by  manipulating  the  input  data  to 
produce  a given  mean  MLSC,  and  then  examining  the  vari- 
ance and  type  of  distribution  at  each  of  say  ten  points. 

If  it  can  be  seen  that  the  type  of  distribution  is  in- 
variant, and  that  the  variance  changes  in  a linear  fashion, 
then  the  variance  for  any  point  on  the  scale  can  be  found 
by  conducting  a linear  regression  on  the  ten  existing  data 


109 


OOR/SM/75D-6 


points.  A second  method  of  predicting  the  variance  at  any 
• point  on  the  scale  is  to  assume  a constant  mean  to  variance 
ratio.  This  method  has  been  found  to  be  leas  accurate  than 
the  linear  regression,  however. 

If  there  is  a need  to  investigate  test  length,  then 
that  variable  must  be  introduced  at  this  point.  The  con- 
ditional distributions  of  MLSC  must  be  established  over  the 
range  of  test  lengths  of  interest. 

Next  the  decision  curves  can  be  constructed.  A 
decision  curve  is  defined  in  general  here  as  a curve  which 
shows  the  conditional  probability  that  action  x should  be 
taken,  that  is,  the  probability  that  action  X should  be 
taken,  given  an  observation  of  MLSC.  Recall  thav  in  general 
such  curves  may  be  constructed  for  more  than  one  obser- 
vation of  MLSC.  Figure  23  depicts  the  decision  curves  for 
the  F-16  study  as  a function  of  test  lengths  from  1,000  to 
10,000  hours.  If  the  problem  is  just  to  determine  what 
degree  of  uncertainty  exists  for  a decision  based  on  an 
observation  of  MLSC  for  various  test  lengths  then  the  task 
is  complete. 

If  the  problem  of  interest  is  a more  fundamental  one 
of,  for  example,  determining  what  the  decision  criterion 
should  be  then  some  additional  analysis  is  required.  Con- 
sider the  case  of  the  F-16  invocation  ratio.  Suppose  for 
a moment  that  this  decision  (the  COD  invocation  ratio)  is 
still  to  be  made;  and  further  suppose  that  the  verification 
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Figure  23.  Lecision  Curves  as  a Function  of  Test  Length 


teit  length  may  be  varied.  Suppose  that*  there  wee*  a 
proposal  for  a COD  invocation  ratio  of  1„1,  or  ten  per  cent 
above  TLSC . Xn  this  example  assume  that  t her contractor 
indicates  a desire  for  a higher  invocation  ratio  and  a 
willingness  to  reduce  the  price  of  the  COD  ^guarantee  at  a 
rate  of  '$267,000  per  “percentage  point  to  an  imtocation 
ratio  of  1.25.  The  contractor  is  implioitly  stating  that 
the  statistical  risk  of  erroneous  invocation,  which  will  be 
called  B^ta,  is  too  great  at  the  1.1  ratio.  Xn  this  sit* 
uation.  Beta  is  found  to  be  about  17  per  cent.  . The  Date 
for  the  invocation  ratio  tit  1.25  is  found  to  be  slightly 
less  then  l.'O  per  'cent.  An  imputed  value  can  now  be  placed 
on  each  percentage  point  of  risk  in  the  perception  of  the 
contractor.  Since  the  contractor  is  willing  to  pay 
$4,005,000  for  a 16  per  cent  reduction  in  risk,  it  is 
apparent  that  the  imputed  value  of  that  risk  reduction  is 
approximately  $250,000  per  percentage  point  of  Beta. 

Suppose  now  that  the  level  of  risk  desired  by  the 
contractor  can  be  provided  by  adjusting  the  test  length  as 
well  aS  the  invocation  ratio.  And  further  suppose  that 
the  government  has  a desire  to  hold  the  invocation  ratio 
at  1.1.  Then  the  requirements  of  both  parties  can  in 
theory  be  satisfied  in  two  ways:  1.  Pay  the  contractor 

the  price  of  increased  risk.  2.  Reduce  the  risk  through 
increased  test  length  (and  absorb  the  additional  testing 
costs . ) 
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From  a chart  similar  to  Figure  23  it  can  be  deter- 
mined that  a 10,000  hour  test  would  reduce  the  Beta  to 
about  9.8  per  cent  for  an  invocation  ratio  of  1.1. 

Assuming  a linear  relationship  then  it  can  be  said  that 
each  900  hours  of  test  time  decreases  Beta  by  one  per  cent. 
Extrapolating  beyond  10,000  hours,  it  can  be  determined 
that  a Beta  of  one  per  cent  will  be  attained  at  17900  hours 
of  testing.  Certainly,  this  is  an  extraordinary  test 
length  and  it  is  unlikely  such  a lengthy  test  would  be 
undertaken.  For  more  reasonable  levels  of  Beta,  say  10  to 
15  per  cent,  much  more  reasonable  test  lengths  would  be 
expected. 1 

To  carry  the  above  analysis  one  step  further,  it 
would  be  possible  in  this  situation  for  the  government  to 
compare  the  cost  of  buying  the  COD  guarantee  at  the  3,500 
hour  Beta  level  with  the  cost  of  testing  to  reduce  the 
Beta  to  one  per  cent.  The  contractor  has  already  (im- 
plicitly) stated  that  each  one  per  ^ent  reduction  in  Beta 
will  bring  a $250,300  reduction  in  COD  guarantee  price. 
Equating  900  hours  per  percent  to  $250,300  per  percent  it 
is  apparent  that  if  the  price  of  testing  is  less  than  $278 
per  flying  hour,  then  it  would  be  less  expensive  to  in- 
crease test  length  than  to  pay  for  the  higher  Beta  risk. 


*An  interpolation  or  extrapolation  such  as  demonstrated 
here  should  be  undertaken  with  some  caution,  as  the 
assumption  of  linearity  is  questionable. 
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Um  of  the  Model  for  Analysis 
of  Contractor  Strategy 

The  simulation  methodology  developed  here  can  to  used 
quite  conveniently  in  evaluating  the  feasibility  of  various 
contractor  strategies  both  before  and  after  the  fact  of  con- 
tract negotiation.  The  example  presented  here  is  after  the 
fact. 

Suppose  in  the  current  contract  situation  a signifi- 
cant underrun  of  MTBF's  were  to  occur.  In  particular,  sup- 
pose the  MTBF's  were  only  slightly  better  than  those  of  the 


1 


previous  or  baseline  equipment  upon  which  the  forecasts  were 
based.  This  will  be  referred  to  as  a condition  of  "minimum 
technological  improvement . " The  values  of  MTBF  to  be  used 
for  the  minimum  technological  improvement  were  decided  in  a 
conference  with  Mr.  Perry  Stewart  (2).  The  values  usea  are 
shown  in  Table  X along  with  the  originally  predicted  values 
Of  MTBF. 


TABLE  X 


MINIMUM  TECHNOLOGICAL  IMPROVEMENT  SCENARIO 


FLU 

Minimum  Technological 
Improvement  MTBF 

Original 

MTBF 

4UD 

100 

1 IT”' 

Navigation  unit 

ITS 

5co 

fire  Control  Computer 

275 

428 

Electronics,  HUD 

215 

385 

'‘light  Control  Computer 

144 

144 

ladar  EO  Display 

do 

nrs  < 

Mgital  dean  Converter 

156 

274 

Jlectrdhics  teo 

1§8 

IfB  ~ 

i Hughes  Radar  Components  S 

teceived/Exciter 

220 

3T<5 

>ata  Processor 

HF5 

274 

Signal  Processor 

r~  ido  ■' 

51T3 

Transmitter 

144 

kntenna 

IT6 

Sl5 

) 
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The  average  MTBF  predicted  under  the  minimum  technological 
improvement  scenario  is  about  .6  of  that  which  is  predicted 
in  the  contract.  After  considering  the  historical  experi- 
ences in  MTBF  predictions  discussed  in  Chapter  1,  this 
scenario  does  not  seem  to  be  one  which  is  totally  outside 
the  realm  of  possibility.  Consider,  for  example,  the 
latest  AFM  66-1  data  on  field  MTBF  for  the  baseline  items 
for  the  first  two  FLUs  above.  The  A7D  MUD  is  demonstrating 
a field  MTBF  of  38  hours.  The  inertial  navigation  unit,  a 
field  MTBF  of  45  hours  (3). 

Assume  then,  that  those  MTBFs  which  the  equipment 
appears  to  be  capable  of  delivering  at  the  time  of  initia- 
tion of  the  3,500  hour  verification  test  are  as  shown  in 
Table  X.  Given  these  conditions,  what  is  the  contractor's 
ability  to  control  the  outcome  of  the  logistics  support- 
ability  evaluation  tests?  This  question  assumes  as  well 
that  the  contractor  is  able  to  estimate  those  approximate 
values  of  MTBF  which  will  be  delivered. 

There  is,  of  course,  only  one  input  which  is  subject 

to  verification  which  can  be  directly  manipulated  by  the 

contractor  without  changes  in  hardware.  That  is  the  unit 

2 

cost  of  each  FLU.  The  question  then,  is  how  much  leverage 
can  be  exercised  through  control  of  FLU  unit  cost  to  correct 

2 

The  tern  Unit  Cost  (UC)  which  will  be  used  in  the  cost 
equations  for  the  verification  test  refers  to  the  average 
unit  cost  of  the  FLU  in  effect  during  the  period  of  the 
verification  test. 
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for  high  logistics  support  costs  which  are  resulting  from 
low  MTBF's. 

Under  the  conditions  described  above,  with  the  true 
MTBF's  equal  to  those  described  under  minimum  technological 
improvement  and  no  other  changes  to  any  inputs,  the  mean  of 
the  distribution  of  measured  logistic  support  costs  would  be 
$61.6  million.  The  likelihood  that  one  observation  of  MLSC 
would  result  in  an  invocation  of  COD  under  these  conditions 
is  greater  than  .99.  Suppose  the  contractor  could  afford 
to  reduce  the  average  unit  prices  of  each  FLU  to  say  70  per 
cent  of  the  original  predicted  price.  This  would  result  in 
a mean  MLSC  of  48.0  million,  and  a likelihood  of  invoking 
COD  of  about  .5. 

Now  suppose  the  contractor  desired  a small  riBk  of 
invoking  COD,  say  on  the  order  of  .10.  This  amount  of  risk 
could  be  established  by  reducing  unit  cost  per  FLU  to  .625 
of  the  predicted  value. ^ 

Although  it  seems  somewhat  unlikely  that  award  fees 
would  be  a matter  for  consideration  in  this  contractor 
strategy,  it  is  interesting  to  note  that  if  the  unit  cost 
of  each  FLU  is  reduced  to  .50  of  the  predicted  value,  the:* 
the  likelihood  of  receiving  an  award  is  .36  while  the  like- 
lihood of  having  COD  invoked  is  virtually  zero.  If  the 

^Tfcese  values  were  derived  by  uniformly  reducing  the  cost 
per  FLU  by  the  given  factor.  The  resulting  probability 
distribution  of  MLSC's  was  then  analyzed  in  order  to  derive 
the  conditional  probability  of  these  events. 
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entire  award  were  to  be  given  for  an  MLSC  <_  TLSC  then  the 
expected  value  of  the  award  for  this  (average  unit  cost 
equal  .5)  position  would  be  $760,000.  (.38  x 2.0  million). 

As  a result  of  this  brief  investigation  of  a con- 
tractor strategy  two  statements  can  be  made. 

1.  The  simulation  methodology  developed  here  is  a 
convenience  tool  for  analysis  of  contractor  strategy. 

2.  In  the  cast?  at  hand,  it  can  be  seen  that  the  F-16 
contractor  can  exercise  considerable  control  over  the  out- 
come of  the  LSC  test,  even  under  a minimum  technological 
improvement  scenario.  In  fact,  the  contractor  can,  by 
manipulating  unit  cost,  establish  virtually  any  desired 
level  of  risk  of  invocation  of  COD.  It  is  beyond  the 
scope  of  this  thesis  to  investigate  the  relative  economic 
advantage  to  the  contractor  of  this  kind  of  strategy.  It 
is  worth  considering  though,  that  such  a strategy  of  avoid- 
ing COD  might  result  more  from  a desire  to  prevent  damage 
to  the  contractor' 8 reputation  and  future  potential  than 
from  pure  short  run  economic  considerations  (54:321). 

Award  Fee  Designs 

In  this  section,  the  simulation  methodology  will  be 
exercised  in  formulating  an  award  fee  design.  The  partic- 
ular award  fee  design  presented  in  this  analysis  will  be 
applicable  to  the  F-16  control  FLU  award  fee  in  the  amount  of 
$2,000,000.  The  methodology,  or  some  variation  thereof, 
can  readily  be  adapted  to  employment  in  other  similar 
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award  faa  applications.  The  fundamental  prerequisite  to 
the  application  of  this  methodology  is  that  the  amount  of 
the  award  fee  be  unilaterally  determined  by  the  government 
based  only  upon  the  attainment  of  some  measurable  logistics 
support ability  goal. 

Considerations  in  Formulating  a Design 

There  are  two  general  statements  which  can  probably 
be  made  about  formulation  of  any  award  fee  plans  based  on 
logistics  supportability. 

First,  any  plan  must  be  objective  enough  to  avoid 
even  the  appearance  of  arbitrary  or  capricious  determination 
of  the  fee. 

i 

Second,  the  major  consideration  in  evaluation  for 
the  award  fee  must  be  the  contractor's  pei formance  in 
minimizing  logistic  support  cost. 

A third  consideration  ought  to  apply  in  cases  where 
there  is  statistical  uncertainty  involved  in  the  measure- 
ment of  goal  attainment.  This  is,  simply  that  the  uncer- 
tainties ought  to  be  considered  in  the  determination  of 
the  fee. 

With  this  last  point  in  mind,  the  methodology 
described  here  will  construct  an  award  fee  curve  which 
is  based  on  the  "amount  of  confidence"  that  an  award  is 
truly  deserved.  The  "amount  of  confidence"  factor  will 
be  applied  to  a somewhat  artificial  sharing  ratio  which 
will  in  turn  determine  the  absolute  amount  of  the  award 
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for  any  given  value  of  MLSC. 

Precedent 

There  is  no  exact  precedent  for  this  award,  but  there 
have  been  applications  which  bear  some  resemblance  to  this. 
In  particular,  the  ARC-XXX/ARC-164  program  employed  an 
incentive/penalty  contract  which  was  based  on  measured 
logistic  supportabil ity.  The  ARC-164  program  actually 
established  the  incentive  as  an  acquisition  cost  adjust- 
ment, but  the  application  is  still  sufficiently  analagous 
to  be  of  interest  here.  For  the  ARC-164  the  following 
award  arrangement  was  established: 

.iWhere:  MLCC  is  measured  life  cycle  cost: 

And  TLCC  is  target  life  cycle  cost 
If  |MLCC-TLCC|  < .03  TLCC;  no  award 
If  MLCC  < .97  TLCC  then; 

Award  - .5  (.97  TLCC-MLCC) 

Stated  verbally,  a three  per  cent  "dead  spot"  was  estab- 
lished to  allow  for  statistical  uncertainty.  Beyond  that 
point  a 50/50  sharing  ratio  was  established,  with  the  con- 
tractor receiving  50  per  cent  on  every  dollar  that  he  was 
able  to  reduce  MLCC  up  to  the  maximum  potential  award 
fee  (53:15). 

The  Methodology 

The  methodology  suggested  here  incorporated  the 
ideas  above  and  one  additional  concept.  The  additional 
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concept  is  that  sor.e  non  zero  amount  of  awar  j ought  to  be 
ipade  if  the  measured  logistic  support  cost  i.  .•  jual  to  the 
target  logistic  support  cost.  In  other  words,  there  is 
no  Mead  spot"  in  this  design. 

Using  only  three  premises  then,  it  is  possible  to 
construct  a reasonable  award  curve.  The  premises  atet 

1.  Maximum  sharing  ratio  is  50  per  cent. 

2.  Dollar  amount  of  award  increases  with  the 
probability  that  LSC  is  truly  less  than  TLSC. 

3.  Some  positive  dollar  amount  of  award  is  made 
at  MLSC  - TLSC. 

The  generalized  award  fee  design  depicted  in  Figure 
24  was  constructed  using  these  three  rules. 

The  following  procedures  were  used  in  actually 
formulating  the  design. 

The  maximum  award  is  given  at  that  point  where  MLSC 
is  two  standard  deviations,  2o,  below  TLSC.  This  point, 

2o  below  TLSC,  is  the  point  where  it  is  virtually  certain 
that  true  LSC  is  below  target  LSC.  That  is,  it  is  that 
value  of  an  observation  which  would  result  in  a 98  per  cent 
left-hand,  one-sided  confidence  interval  whose  right  end 
point  is  < TLSC.  To  see  this,  note  that 


P (y  < TLSC)  * .98 


- P 


, -MLSC  + u 
1 a (ML8C) 


< 


TLSC  - MLSC, 
0 (MLSC)  ' 


- P 


,MLSC  - u 
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„ MLSC  - TLSC 
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Since 


MLSC-u 

o(MLSC) 


is  a standard  normal  deviate,  Z, 


we  have 


P (2  > 


MLSC  - TLSC 
0 (MLSC) 


.98 


9 


which  suggests  that 

,MLSC*  - TLSC 
' oMLSC 


Z 


02 


« 2.0 


so  that 

TLSC -MLSC*  = 2.0  o(MLSC) 
or 

MLSC*  ■ TLSC  - 2.0  o(MLSC). 

The  maximum  sharing  ratio  of  50/50  is  maintained  to  a point 
which  is  1.5  standard  deviations  below  TLSC.  Over  this 
entire  range,  from  2o  to  l.So  the  average  probability  that 
true  LSC  is  below  TLSC  is  approximately  .96. 

Over  the  range  of  MLSC’s  between  l.Oo  and  1.5o 
below  target  the  sharing  ratio  is  reduced  to  70/30 
(Government/contractor).  Over  this  range  the  average 
probability  is  .89. 
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Over  the  range  of  MLSC's  which  are  between  .So  and 
l.Oo  below  target  the  sharing  ratio  is  reduced  again  to 
90/10.  Over  this  range  the  average  probability  that  MLSC 
is  less  than  TLSC  is  .77. 

Finally  then  over  the  range  of  MLSC's  between  O.Oo 
and  .So  the  sharing  ratio  is  reduced  to  zero.  Hence, 
starting  at  MLSC«TLSC,  motivation  to  reduce  MLSC  increases 
as  MLSC  gets  smaller  since  a greater  fraction  of  LSC  savings 
goes  to  the  contractor. 

In  order  to  adapt  this  design  to  some  situations,  it 
may  be  necessary  to  adjust  the  sharing  ratios  of  the  two 
center  quarters  depending  upon  the  relative  magnitude  of 
the  award  and  the  logistic  support  cost  being  measured.  In 
some  cases  it  might  be  desirable  to  use  a division  of  other 
than  one  half  a standard  deviation. 

Figure  25  shows  the  application  of  this  award  fee 
design  to  the  F-16  control  FLUs.  The  standard  deviation 
used  here  was  for  ail  three  inputs  stochastic,  time  to 
failure,  manhours,  and  fraction  repaired  this  station.  If 
only  MTBF  were  to  be  considered  stochastic  then  the  maxi- 
mum award  would  occur  at  a higher  value  of  MLSC. 

It  is  surely  apparent  that  such  a dogmatic  scheme  as 
this  if  accepted  would  leave  little  room  for  interpretation 
on  the  part  of  the  fee  determining  official.  Certainly 
such  a plan  can  be  expected  to  provide  nothing  more  than  a 
well  reasoned  guide  to  bolster  the  intuition. 

If  the  curve  is  to  be  adjusted  before  measurement  of 
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LSC,  to  account  for  subjective  considerations,  then  it  would, 
be  reasonable  to  move  the  entire  curve  in  a vertical  (rather 
than  horizontal)  manner.  This  would  provide  an  adjustment 
in  the  base  amount  of  the  award  fee  for  MLSC  * TLSC  and  an 
equal  adjustment  for  any  given  observation  of  MLSC.  Of 
course,  an  increase  in  the  base  award  will  provide  for  the 
full  award  to  be  given  at  a higher  value  of  MLSC.  For 
example,  if  the  base  award  is  raised  to  $1,000,000  the 
maximum  award  will  occur  at  32.2  million  MLSC  or  at  approxi- 
mately 1.6  standard  deviations  below  the  target  logistic 
support  cost.  Such  a point  (1.6o  below  the  target)  would 
still  provide  a confidence  level  of  about  94  per  cent  that 
the  true  LSC  is  below  TLSC. 

Limitations 

The  award  fee  design  described  above  is  a product  of 
all  of  the  assumptions  of  both  the  logistic  support  cost 
model  and  the  simulation  methodology  described  herein. 

It  is  most  directly  dependent  upon  the  probability  distri- 
bution of  MLSC.  If  an  error  is  made  in  specifying  the  form 
of  the  distribution  of  MLSC’s,  considerable  error  would  be 
introduced  into  the  methodology.  It  has  been  determined 
for  example,  that  if  the  time  to  repair  distributions 

have  a Weibull  distribution  with  a small  shape  parameter, 

•'  ‘ \ . ■ 

say  .18,  then  the  distribution  of  MLSC's  will  have  a far 

t 

' 1 • 

greater  variance  than  under  other  distributions.  It  hrr ' 
also  been  determined  that  the  distribution  of  MLSC's,  with 
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a Neibull  time  to  repair  distribution,  will  have  a highly 
skewed  Weibull-type  distribution.  If  then,  in  some  appli- 
cation of  this  methodology,  it  were  determined  that  the 
manhour  inputs  had  a weibull  distribution,  it  couxd  be 
expected  that  the  distribution  of  MLSC's  would  be  non- 
normal. 

It  should  also  be  noted  here  that  in  any  application 
of  this  award  fee  methodology,  the  actual  amount  of  award 
received  will  be  dependent  on  the  verification  test  length. 
Since  an  increase  in  test  length  reduces  the  standard 
deviation  of  the  MLSC,  for  any  given  value  of  MLSC  TLSC, 
a greater  award  will  be  given  for  a greater  test  length. 
This  fact  might  increase  contractor  motivation  to  seek 
greater  test  lengths. 

As  a final  point  it  should  be  noted  once  again  that 
the  logistic  support  cost  used  here  is  not  inclusive  of  all 
support  costs.  Rather,  it  is  as  described  in  Chapter  1,  a 
measurable  standard  against  which  to  compare  relative 
performance.  This  fact  calls  into  question  the  concept  of 
a sharing  ratio.  The  question  is:  is  it  meaningful  to 

establish  a sharing  ratio  in  terms  of  a measurement  which 
does  not  include  all  relevant  costs?  In  fact,  it  car.  also 
be  argued  that  the  measurement  includes  some  costs  which 
are  not  relevant;  for  example,  some  would  argue  that 
maintenance  manhour  costs  are  not  relevant.  In  support 
of  the  methodology  used  here  then,  it  can  be  said  that; 
even  acknowledging  the  weaknesses  of  the  data,  the  measured 
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logistic  support  cost  determined  in  the  3,500  hour  test  is 
the  only  verified  logistic  support  cost  in  existence.  In 
this  light  it  seems  appropriate  to  establish  a sharing  ratio 
based  on  this  known  cost. 

The  three  applications  presented  in  this  chapter 
should  give  the  reader  a general  idea  of  the  utility  of 
the  methodology.  Certainly  the  usefulness  of  the  model 
is  not  limited  to  these  particular  applications.  To 
facilitate  the  use  of  this  model  in  other  investigations, 
e generalization  of  the  methodology  is  included  in  the 
final  chapter  along  with  the  conclusions  and  recommendations 
for  further  study.  _ ~ «.  : „v. ,,  : 
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Chapter  6 

SUMMARY , CONCLUSIONS,  AND  RECOMMENDATIONS 
FOR  FURTHER  STUDY 

Introduction 

This  chapter  is  presented  as  a summary  and  as  a 
generalization  of  the  methodology  so  that  it  might  readily 
be  adapted  by  the  reader  to  other  applications  of  logistic 
supportability  measurement  and  evaluation.  Also  included 
here  are  the  conclusions  and  recommendations  for  further 
study. 

Data 

Data  collection  for  input  to  the  simulation  model 
may  be  a relatively  simple  matter  if  the  applicable  con- 
tract has  been  drawn  up  and  if  such  data  as  MTBF,  MMH,  RTS, 
RIP,  etc.  are  contractually  specified.  The  credibility  of 
the  contractual  specifications  can  only  be  evaluated  on 
a case  by  case  basis,  and  for  those  data  elements  which 
are  subject  to  verification,  the  credibility  will  evan- 
tually  be  determined  by  testing.  Data  collection,  or  more 
correctly,  data  forecasting  for  systems  in  early  develop- 
ment phase  must  be  based  on  AFM  66-1  data  from  similar 
systems. 1 

^For  a discussion  of  data  collection  problems  and  some  new 
data  collection  systems  see  Estimation  of  Life  Cycle  Coats: 
A Case  Study  of  the  A7D  by  Marco  Fiorello  (23) . 
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The  LSC  Model,  Choice  of  Terms 

In  tailoring  the  LSC  model  to  a specific  application, 
it  must  be  realized  that  in  some  cases  inclusion  of  fewer 
terms  may  provide  greater  accuracy.  Certainly,  thus  would 
be  true  when  the  data  inputs  for  some  of  the  terms  are 
particularly  weak.  The  decision  as  to  which  terms  of  the 
LSC  mode),  to  employ  can  only  be  made  on  a case  by  case 
basis  after  consideration  of  the  relative  importance  of  each 
term  to  the  decision  at  hand  and  the  ease  and  accuracy  with 
which  it  can  be  verified. 

Use  of  Monte  Carlo  Methods 
or  Analytical  Methods 

Analytical  methods  may  be  feasible  in  some  applications. 
If,  for  example,  it  is  necessary  to  examine  the  variance  of 
MLSC  as  a function  of  only  one  input  random  variable,  then 
it  may  be  possible  to  use  propagation  of  errors  techniques. 

It  is  unlikely  that  convolution  integrals  would  be  useable 
in  analyzing  anything  so  complex  as  a life  cycle  cost  model. 

In  considering  the  use  of  propagation  of  errors  techniques, 
it  is  important  to  recall  that  each  of  the  input  distribu- 
tions must  be  normal. 

The  Monte  Carlo  Method 

In  subsequent  discussion,  it  will  be  assumed  that 
the  Monte  Carlo  method  is  being  employed;  much  of  the 
development  would  follow  a precisely  parallel  path  in 
analytical  methodology.  . 
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After  decisions  have  been  made  as  to  which  of  the  LSC 
model  parameters  should  be  subject  to  verification)  and 
among  this  group  which  should  be  treated  as  random  vari- 
ables,  then  the  next  step  is  to  select  probability  dis- 
tributions for  these  random  variables.  As  has  been  shown, 
theory  is  a powerful  tool  in  making  this  selection  when 
faced  with  a paucity  of  relevant  data.  Data  are  available 
on  distributions  of  time  to  repair  and  time  to  failure  on 
certain  equipments.  These  data  have  been  collected  and 
fitted  to  theoretical  distributions  during  Category  II 
reliability  and  maintainability  testing  at  the  AF  Flight 
Test  Center.  In  using  these  data,  caution  should  be 
exercised  to  attempt  to  find  equipments  which  are  similar 
in  function  and  construction. 

If  an  actual  empirical  distribution  on  a particular 
equipment  is  available,  it  can  be  incorporated  into  the 
simulation  model.  In  the  usual  case,  however,  it  will  be 
necessary  to  employ  a theoretical  distribution.  Conversion 
of  the  theoretical  distribution  into  a Monte  Carlo  process 
generator  is  a relatively  simple  matter  as  illustrated  in 
the  appendix  for  exponential,  Weibull,  and  lognormal  dis- 
tributions. 

In  some  cases,  the  determination  of  the  correct 
probability  distribution  may  not  be  very  clear.  In  such 
situations  it  is  advisable  to  incorporate  the  "second  best" 
distribution  into  the  model  in  order  to  determine  whether 
the  difference  in  inputs  will  have  a significant  impact 
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oij  the  output.  As  demonstrated  in  this  study,  the  incor- 
poration of  a Weibull  distribution  of  failures  \with  shape 
parameter  l.L)  caused  an  insignificant  change  in  output. 

The  incorporation  of  a Weibull  distribution  of  time  to 
repair  (with  shape  parameter  .18)  however,  caused  an  extra- 
ordinary ohange  in  output  variation.  The  lessons  learned 
from  this  sensitivity  analysis  are: 

1.  It  is  not  necessary  to  be  particularly  concerned 
about  whether  the  time  to  failure  is  more  accurately 
represented  by  the  exponential  or  Weibull  distribution. 

2.  It  is  important  to  determine  as  conclusively  as 
possible  whether  the  time  to  repair  of  a given  item  should 
be  lognormal  or  Weibull.  If  sufficient  data  were  avail- 
able, consideration  could  be  given  to  representing  each 
FLU  by  its  own  appropriate  distribution. 

Before  leaving  the  subject  of  sensitivity  analysis, 
the  subject  of  experimental  design  should  be  discussed, 
since  the  experimental  design  in  this  context  is  just  a 
highly  formalized  sensitivity  analysis.  When  more  than 
two  factors  are  involved  in  a sensitivity  analysis, 
intuitive  methods  fail  and  the  framework  of  experimental 
design  is  necessary.  In  the  case  at  hand  four  factors  were 
involved  (MTBF,  MMH , RTS,  and  test  length).  Each  factor 
had  two  levels.  Also,  interactions  were  expected  among 
certain  of  the  terms.  Virtually  any  number  of  factors  with 
up  to  three  factor  levels  each  can  be  handled  by  the  methods 
of  references  15,  42,  and  47. 
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With  the  form  of  the  input  parameters  determined  and 
incorporated , the  next  step  is  the  determination  of  the 
form  and  parameters  of  the  output  distribution  (distri- 
bution of  MLSC's).  Three  items  of  information  are  indis- 
pensable in  this  determination  and  ought  to  be  included  as 
a part  of  the  computer  print  out.  These  items  are  the 
mean  and  standard  deviation  of  the  distribution  and  a 
histogram.  The  histogram  is  essential  in  forming  an 
educated  null  hypothesis  as  to  what  the  distribution  appears 
to  be.  The  mean  and  standard  deviation  are  required  to  form 
estimators  of  the  hypothesized  distribution's  parameters. 

The  Kolmogorov-Smirnov  (K-S)  test  can  be  used  at  this 
point  in  accepting  or  rejecting  the  distribution  specified 
in  the  null  hypothesis.  If  the  hypothesized  distribution 
is  non-normal,  its  parameters  may  have  to  be  estimated  by 
the  method  of  moments  (42:300).  If  the  hypothesized  dis- 
tribution is  normal,  then  of  course  the  computer  output 
of  mean  and  standard  deviation  are  the  estimators  of  the 
parameters.  For  a normal  distribution  the  K-S  test  may 
be  conducted  using  the  normal  tables.  For  other  dis- 
tributions, it  may  be  necessary  to  find  the  points  of  the 
cumulative  density  function  by: 

F(Y)  - fY  f (t)  dt 

wOO 

where  f(t)  is  the  hypothesized  probability  density  function. 

The  last  step  in  the  generalized  methodology  is 
determination  of  the  form  of  the  probability  distribution 
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of  KLBC's.  The  next  step  in  the  analysis  will  be  dependent 
upon  the  particular  application. 

Conclusions 

One  of  the  stimuli  for  this  thesis  was  a study  by  the 
RAND  Corporation  which  questioned  the  deterministic  use  of 
the  AFLC  Logistics  Support  Cost  Model  as  a tool  for 
acquisition  decision  making  (62) . The  hypothesis  suggested 
by  that  study  was  that  the  AFLC  Logistic  Support  Cost  Model 
might  be  of  greater  utility  in  a stochastic  form.  This 
study  definitely  supports  that  hypothesis.  It  has  been 
shown  in  this  study  that  a knowledge  of  the  statistical 
variance  of  the  Measured  Logistic  Support  Cost  can  be  use- 
ful in  making  decisions  about  contractual  incentives 
and  verification  test  length.  It  can  be  seen  that  the  un- 
certainty (standard  deviation)  of  the  Measured  Logistic 
Support  Cost  is  not  so  large  as  to  create  unacceptable  un- 
certainties in  most  situations. 

Depending  upon  what  assumptions  are  made,  it  would  be 
possible  to  state  that  the  standard  deviation  of  the  MLSC 
is  between  7%  and  29%  of  the  mean.  The  7%  figure  is  for 
13  FLU' 8 with  only  MTBF  stochastic,  while  the  29%  figure 
is  for  one  FLU  with  three  stochastic  inputs;  MTBF,  MMH 
and  RTS.  A meaningful  average  figure  might  be  19%  of  the 
mean  which  is  for  13  FLU's  with  three  stochastic  inputs. 

This  figure,  (19%  of  the  mean)  might  be  called  represen- 
tative because  it  is  likely  that  in  future  tests  of  aircraft 
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logistic  support  cost  there  will  be  some  relatively  small 
group  of  control  FLU's.  It  is  likely  that  verification 
will  be  required  for  at  least  MTBF,  MMH,  and  RTS,  thus 
requiring  stochastic  representation  of  these  inputs  in 
the  model.  It  can  be  seen  from  Appendix  A that  the 
standard  deviation  to  mean  ratio  varies  with  the  number 
of  FLU' 8 tested,  at  least  over  the  range  between  one  and 
25  FLU's.  It  would  be  of  interest  in  further  studies  to 
determine  the  behavior  of  this  ratio  over  a greater  range. 

The  hypothesis  stated  in  the  first  chapter  of  this 
study;  namely  that  the  major  contributors  to  uncertainty 
would  be,  in  descending  order  of  importance,  MTBF,  MMH, 
and  RTS  has  been  disproven.  In  fact,  the  study  indicates 
that  the  contributions  to  uncertainty  of  MTBF  and  RTS  are 
about  equal  while  the  contribution  of  MMH  is  secondary. 

The  practical  benefits  of  this  study  have  been  shown 
in  the  three  different  applications  demonstrated  herein. 
These  are;  determination  of  suitable  test  length  and  COD 
invocation  ratios,  contractor  strategy  investigation,  and 
award  fee  design.  The  methodology  is  useful  as  well  in 
any  investigation  which  involves  prediction  and  verifi- 
cation of  Life  Cycle  Cost. 

Recommendations 

An  area  which  appears  to  be  a potentially  fruitful  one 
for  further  researcn  into  Life  Cycle  Cost  Prediction  and 
Verification,  is  that  of  test  length  optimization. 
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■As  an  area  wSoie  At  is  likely  the  simulation  methodology 
Sfcown  hare  could  ba  profitably  used. 

In  particular;  if  the  benefits  of  increased  test 
lengths  could  be  compared  with  the  costs  of  increased 
testing,  then,  in  theory  at  least,  a test  length  could  be 
found  which  would  minimise  the  total  cost  of  testing.  The 
first  problem  which  arises  here  is  prediction  of  the  vari- 
able  costs  of  testing.  One  method  of  estimating  these 
costs  is  a parametric  Cost  Estimating  Relationship  (CER) 
published  in  a previous  study  (41:9-2) 

Here  the  cost  of  the  L5C  analysis  in  Dollars  is: 

C - (500) (A) (N*93) 

where  N is  the  number  of  FLU's,  and  A is  an  ex- 
perience factor  with  a mean  value  of  .65. 

This  equation  reportedly  predicts  the  cost  of  performing 
a typical  support  cost  analysis,  that  is,  testing  and 
collection  of  the  data  required  to  input  to  the  Logistic 
Support  Cost  Model.  Two  problems  arise  in  attempting  to 
apply  this  CER  in  this  application.  First,  the  definition 
of  typical  is  not  given.  Without  some  way  of  determining 
what  is  the  typical  tost  length  considered  is  this  CBR,  it 
is  impossible  to  use  it  in  estimating  the  costs  of  testing 
over  various  fixed  length  tests. 

The  second  problem  is  that  the  equation  predicts 
such  small  values  for  testing  cost  as  to  be  simply  not 
believable.  For  example,  assume  a very  small  value  of 
typical  test  length,  say  500  hours.  Based  on  this 
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assumption*  the  equation  would  predict  a cost  of  about 
$32,000  for  testing  13  FLU's  over  3500  hours.  This 
seems  an  extremely  low  estimate. 

Assuming,  though,  that  a credible  prediction  of  the 
variable  costs  of  testing  can  be  made  by  a CER  or  other 
methods,  then  some  hourly  cost  can  be  assigned  for  LSC 
testing. 

Determination  ("»f  the  benefits  of  testing  is  somewhat 
more  complex.  The  benefits  must  be  measured  in  terms  of 
decision  costs.  That  is;  how  much  improvement  in  decision 
reliability  results  from  each  hour  of  increased  testing? 

An  increase  in  decision  reliability  then  can  be  described 
as  a decrease  in  the  costs  associated  with  bad  decisions. 

Now,  what  are  the  costs  of  bad  decisions?  In  a contract 
which  involves  a COD  clause,  one  of  the  most  important  bad 
decision  costs  would  be  an  erroneous  invocation  of  COD. 

This  cost  would,  of  course,  decrease  as  test  length  in- 
creases. Another  bad  decision  cost  would  be  the  cost  of 
not  invoking  COD  when  it  should  be  invoked.  This  cost 
would  also  decrease  as  test  length  increases.  This  con- 
cept can  be  expressed  symbolically  as: 

(6-1)  Bad  Decision  Cost  <BDC>-  /'  <x)dx 

where  x is  a value  of  Measured  Logistic  Support  Cost 

and  f(x)  is  the  PDF  of  MLSC. 

It  is  only  meaningful  to  discuss  these  erroneous  decision 
costs  however,  in  terms  of  some  true  value  of  LSC,  for 
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without  some  true  value  of  LSC  it  is  not  possible  co  say 

what  action  is  correct  or  incorrect. 

* 

Assume  then  for  purposes  of  illustration  that  the 
true  value  of  LSC  is  equal  to  TLSC.  What  then  are  tu" 
bad  decision  costs,  and  how  do  they  vary  with  cesl  length? 
If  true  LSC  equals  TLSC  there  is  only  one  incorrect 
decision  which  can  be  made;  That  is  an  erroneous  in- 
vocation of  COD.  Symbolically,  the  Dad  Decision  Costs 
are: 


1.25  TLSC 


(6-2)  BDC  = / 


fCost  of  failing  t.o  invoke  1 
COD  when  it  should  be  .invoked] 


l 


f (x)  d 


+ 


f 

1.25  TLSC 


Cost  of  Erroneous 
Invocation  of  COD 


f (xlclx 


The  first  term  in  (6-2)  above  is •zero  if  true ■ LSC "TLSC 
The  second  term  can  be  estimated  by  the  following  algorithm. 

First,  assume  True  LSC=TLSC. 

Also,  assume  that  COD  will  be  invoked  whenever  a 
single  observation  of  MLSC  is  greater  than  l.?1.'.  times  TLSC. 

Assume  that,  among  the  13  FLU's  on  the  above  obser  - 
vation,  COD  action  w ill  be  taken  on  whichever  FLU's 
demonstrate  an  MTBF  loss  than  .75  times  the  contract 
specified  MTBF.  Assume  there  are  j FLU's  in  this  category. 

Given  the  above  assumptions,  then  the  costs  to  tbr' 
government  of  COD  may  be  estimated  through  a method 
developed  by  halaban  .5:93;.  Using  this  method,  if  a 
modification  is  made  to  reduce  the  failure  rate,  \,  it 
is  assumed  that  X who-e  M is  a function  of  both 
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the  actual  and  the  specified  failure  rate,  A . Also  M has 
some  minimum  value  M'  so  that  no  modification  can  reduce  A 
to  zero.  M then  is  defined: 


(6-3) 


M-M'  + ( 1-M 1 ) ("SV  /A 


i-M  ' 

* * 

where  M is  the  improvement  factor  expected  at  A=A  . 

Equation  6-3  must  be  i;eratively  applied  to  each  of  the  j 

FLU's  until  each  of  their  is  modified  to  its  specified 

failure  rate.  Assume  there  are  k modifications  required 

for  the  jth  FLU.  The  cost  of  each  modification  is  found 

by:  Cost  of  Modification,  C (M) =1. 06 (e (1“M;/10M-1) Cp  2 

where  C is  the  unit  cost  of  the  FLU.  The  total  cost  of 
P 

modification  can  now  be  found  as: 


Z Z C (M)  . . 
j = l 1=1 


A caution  must  be  included  here.  Balaban  warns  that  this 
model  has  been  de  /eloped  using  very  limited  data  and  is 
included  for  illustrative  purposes.  Its  credibility, 
therefore  is  limited. 

Accepting  this  method  for  the  time  being  however,  as 
the  only  available  estimator,  it  can  be  seen  that  the 
integral  in  6-i  can  be  evaluated.  Realizing  that  the 
variance  of  MLSC  will  change  with  test  length,  this 


tC(M)  is  the  total  cost  of  modification.  It  should  be 
multiplied  by  an  appropriate  factor  representing  the 
government  share  of  modification  costs. 
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integral  should  be  evaluated  for  several  values  of  test 
length.  Hopefully  then  a curve  something  like  Figure  26 
could  be  constructed.  This  would  represent  the  sum  of 
the  variable  testing  costs  and  Bad  Decision  Costs. 

I Total  Cost 


A B C D Test  Length 

Figure  26.  Total  Cost  by  Test  Length 
True  LSC=TLSC  (hypothetical) . 


If  the  curve  has  a minimum  at  a reasonable  test 
length,  then  this  minimum  would  represent  the  optimum 
test  length.  If  the  minimum  occurs  at  a value  of  test 
length  which  is  too  large  to  be  acceptable,  then  a point 
on  the  curve  should  be  chosen  where  the  marginal  return 
is  becoming  very  small,  say  point  B in  Figure  26. 

The  above  procedure  would  derive  a curve  based  upon 
the  assumption  that  True  LSC  equals  TLSC.  It  may  be  of 
interest  to  find  a curve  for  an  assumption  of  True  LSC 
equals  1.5  times  TLSC.  Here  the  second  integral  in  6-2 
would  be  zero  and  the  first  integral  would  determine  rele- 
vant costs.  That  is: 

» 

1.25  TLSC 

BDC=  / ( ( 1 . 5TLSC- (TLSC+Cost  of  Modifications) ) f (x) dx 

— 00 

Bad  decision  costs  here  are  the  difference  between  the  True 
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LSC  and  the  TLSC  less  the  government  cost  of  modification. 
The  integrals  in  both  the  situations  described  above  would 
be  evaluated  through  the  simulation  methodology.  Evaluating 
the  Bad  Decision  Costs  for  several  values  of  True  LSC  might 
give  a series  of  curves  as  depicted  in  Figure  27. 


Assumed  Values  of  True  LSC  (Hypothetical) . 


These  curves  then,  hopefully  could  be  evaluated  for  an 
optimum  or  near  optimum  point. 

The  application  of  the  methodology  developed  in  thin 
thesis  to  the  problems  of  optimizing  test  length  (if  the 
uncertainties  involved  in  the  prediction  of  test  cost  and 
modification  cost  can  be  overcome)  should  prove  to  be  a 
valuable  addition  to  the  instruments  of  Life  Cycle  Cost 
Analysis. 
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ANALYSIS  OF  STANDARD  DEVIATION 
TO  MEAN  RATIO  OF  MLSC 


148 


GOR/SM/75D-6 


This  analysis  was  accomplished  using  the*  Control 
FLU'S  in  the  order  in  which  they  are  listed  in  Table  II 
So;  for  one  FLU,  the  Headup  Display  was  used,  for  two 

FLU's  the  Headup  Display  and  the  Navigation  Unit 
were  used,  etc. 

FLU's  1 through  8 were  the  aircraft  FLU's.  FLU 'a 
8 through  12  were  the  Westinghouse  radar  FLU's.  Beginning 
with  FLU  number  13,  the  Headup  Display  and  other  aircraft 
FLU's  were  introduced  for  a second  time,  with  FLU's  number 
21  through  25  being  the  Hughes  radar  FLU's.  It  appear-', 
that  there  may  be  a minimum  ratio  in  the  vicinity  of  11  to 
13  FLU's.  Further  analysis  is  nexessary  to  determine 
whether  this  minimum  can  be  expected  in  all  applications 
or  whether  it  may  be  a result  of  the  repeated  use  of  the 
eight  aircraft  FLU's  or  other  anomalies  of  the  particular 
data  in  this  application. 
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FLU  MTBF 

UC 

UT 

RIP 

RTS 

PAMH 

HIMH 

RMH 

BMH 

I'M 

Heedup 

Display 

172 

42283 

1. 

0 

.95 

0 

u 

1.0 

4.0 

to . 

Nav igation 
Unit 

200 

116546 

1. 

0 

.95 

• 

0 

l.C 

6.0 

12. 

Fire 

Control 

Computer 

428 

69106 

1. 

0 

.95 

.1 

0 

1.0 

6.0 

6 . 

HUD 

Electronics 

IBS 

31408 

1. 

0 

.95 

.1 

0 

1.0 

4.0 

6. 

Flight 

Control 

Computer 

144 

50332 

1. 

.01 

.98 

.6 

1. 

1 . 

6.0 

12. 

Radar 

EO  Display 
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61840 

1. 

0 

.95 

0 

0 

1 c 

4 . 0 

9 . 

Digital 

Scan 

Converter 

274 

68024 

1. 

0 

.95 

.1 

0 

j . 0 

6.o 

t . 

EO 

Display 

Electronics 

188 

7659 

1. 

0 _ 

.95 

.1 

0 

1.0 

4.0 

KESTINGHOUSE  RADAR  FLU's 

Antenna 

Servo 

St, 

68529 

2 

0 

.9 

0 

0 

.5 

J.v. 

u. 

Low 

Power  RF 

338 

84045 

2 

0 

.85 

0 

0 

.25 

5.0 

Digital 

Processor 

150 

181020 

2 

0 

00 

<r 

• 

0 

0 

.25 

•L  m'J 

;V , 

Trans- 

mitter 

338 

102147 

2 

0 

.3 

0 

0 

.67 

3.0 

8. 

HUGHES  RADAR  FLU 

' s 

Receiver 

Exciter 

340 

46283 

1.3 

0 

.97 

0 

0 

. 33 

1.77 

5. 

Data 

274 

76158 

1.3 

0 

.99 

0 

0 

.25 

1.2 

5, 

Processor 


GOR/SM/75D-6 


FLU 

MTBF 

UC 

UF 

RIP 

RTS 

PAMH 

HIMH 

RMH 

BMH 

DMH 

Signal 

2S2 

89346 

1.3 

0 

.99 

0 

0 

.25 

1.1 

5.0 

Processor 

Trans- 

194 

50298 

1.3 

0 

.92 

0 

0 

.53 

2.04 

8.0 

mitter 

Antenna 

315 

45902 

1.3 

0 

.56 

0 

0 

.83 

1.6 

6.0 

1 r>  3 


Ml 


LSC  million 
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SIMULATION  MODEL,  VARIABLE  NAMES 

BMH  - base  manhours,  contract  specified,  same  as  MMOA 
modal 

BMHM  - measured  base  manhours 

DNH  - depot  manhours,  contract  specified,  same  as  MMOA 
model 

DMHM  - measured  depot  manhours 

HIMH  - in  place  manhours,  contract  specified,  same  as  IMH 
in  MMOA  model 

PAMH  - preparation  and  access  manhourn,  contract  specified 
same  as  MMOA  model 

PAMHM  - measured  depot  manhours 

RMH  - remove  and  replace  manhours,  contract  specified, 
same  as  MMOA  model 

RMHM  - measured  remove  and  replace  manhours 

RTS  - reparable  this  station,  contract  specified,  same 
as  MMOA  model 

RTSM  - measured  fraction  reparable  this  station 

RIP  - reparable  in  place,  contract  specified,  same  as 
MMOA  model 

TBF  - contract  specified  MTBP,  same  as  MTBF  in  MMOA 
model 

TBFM  - measured  MTBF 

RLSC  - measured  Logistic  Support  Cost 


1 


1 

J 


L. 


OOR/SM/75D-6 


The  Computer  Program  and 

Random  Number  Generation  ^ 

V ✓ 

The  computer  program  which  !•  included  in  this  ’.ppen- 
dix  is  a complete  listing  of  the  model  used  in  this  study. 

The  program  has  been  written  using  the  FORTRAN  IV  program- 
ming language.  The  model  was  designed  for  use  in  an  inter- 
acting time  sharing  mode,  as  evidenced  by  the  series  of 
questions  requiring  user  response  to  initiate  the  program. 

When  operated  in  a time  sharing  mode  on  a General  Electric/ 

Honeywell  600  series  computer  system,  both  the  input  and 
the  output  are  at  the  same  remote  terminal. 

The  FORTRAN  programming  is  straightforward  and 
should  be  clear  to  those  familiar  with  the  language.  One 
point,  however,  which  may  cause  some  confusion  is  thr  in- 
clusion of  two  random  number  generators.  The  first  random 
number  generator,  RAND(R) , is  the  primary  generator  and 
the  one  which  was  used  in  this  entire  study.  The  second 
generator  RND(Y),  was  included  as  an  auxiliary  routine  in 
case  the  capacity  of  the  first  generator  was  exceeded. 

Since  approximately  2,600,000  random  number  genera- 
tions are  required  on  the  average,  (For  three  random  input 
variables) , it  can  be  seen  that  a random  number  generator 
with  a short  period  would  eventually  repeat  itself  in  this 
model,  thereby  invalidating  the  results.  To  guard  against 
an  undetected  occurrence  of  repetition,  the  first  20  sorted 
values  of  MLSC  are  printed  out  for  every  simulation  run. 

Exact  duplications  of  the  same  MLSC  are  symptomatic  of 
repetition  in  the  random  number  generator.  Experience 
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with  periodicity  in  this  model  has  shown  that  it  causes 
noticeable  duplication  in  MLSC's  and  a reduction  in  the 
variance  of  the  distribution. 

The  primary  generator  for  this  routine,  RAND(R) , 
is  of  the  multiplicative  congruential  type.  It  utilises 
the  following  recursive  relationship! 

ni+i  = a*ni (mod  P6) 

where  p is  the  number  of  numerals  in  the  computer  number 
system,  2 in  this  case,  and  e is  the  number  of  digits  in 
a word,  in  this  case  35. 

For  this  generator  to  develop  its  maximal  period, 

o 

a must  be  relatively  prime  to  m(m*p  ) , and  a must  be  on 
odd  integer.  Finally,  a must  satisfy  the  following; 
a ® 8t  t 3 

where  t is  any  positive  integer  (33:236). 

For  the  generator  used  in  this  program: 
t = 1200703125  + 3 _ 152587891 

So  that  the  above  recursive  relationship  is  satisfied. 

The  maximal  period  is  also  dependent  upon  the  start- 
ing value,  nQ,  called  the  seed.  This  seed  must  satisfy 
two  conditions: 

1.  nQ  must  be  an  odd  integer 

2.  nQ  =8t+3,  where  t is  a positive  integer 

In  this  simulation  model,  two  seeds  were  used:  317  and  11. 

Both  satisfy  the  above  relationship. 

t = 317  + 3 - 40  t - 11  - 3 - 1 

5 5~ 
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It  has  been  shown  then,  that  the  conditions  for  a maximal 
period  have  been  satisfied  (33:236). 


Derivation  of  Exponential  Lifetimes 
from  Constant  Failure  Rate  Assumption 

As  stated  in  Chapter  4-  the  assumption  of  an  exponen- 
tial distribution  of  lifetimes  derives  inevitably  from  one 
assumption  regarding  the  physical  nature  of  the  equipment. 
That  assumption  is  that  the  equipment  has  a constant  failure 
rate  over  time.  For  this  kind  of  equipment,  chance  alone 
dictates  when  a failure  will  occur  and  the  chronological 
age  of  the  equipment  does  not  affect  the  issue,  "he  a -sump- 
tion of  no  deterioration  over  time,  or  a constant  failure 
rate  is  commonly  made  for  electronic  equipment  (i.,  . C^ven 
this  assumption  then: 

First:  Define  Reliability  R(t)  = l-F(t) 

where  F(t)  is  the  CDF  of  the  appropriate  distribution  of 
failures.  R(t)  is  the  probability  of  survival  at  time  t. 

Define  the  Hazard  rate  Z(t)  as  the  conditional 
failure  rate  function:  Z(t)  = 


where  f(t)  is  the  PDF  of  the  appropriate  distribution  of 
failures.  The  probability  of  survival  from  time  t to 
time  t+At,  given  survival  until  time  t,  is  Z(t)ut. 


And  since  f(t)  is 


dF(t) 


and  R(t) 
then  f(t) 


dR(t) 

~sr~ 

dR(t)/dt 

RfU 


so  that  Z(t) 
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Or  Z(t)  ■ 

t 

and  LnR(t)  * - / Z(t)dt 
o 
t 

-/_  2 (t)dt 
so  R(t)  » e c 

Now  suppose  an  equipment  has  a constant  failure  rate 
or  equivalently  a constant  Hazard  rate. 

Then  Z(t)  * A 

- / 1 Adt 

and  R(t)  = e ° 

= e-At 

F ( t)  = l-e‘At 
dF(t)  = f(t)  = Ae“At 

”3t“ 

And  f(t)  above  is  just  the  probability  density  function  for 
the  exponential  distribution  (12:323). 

Derivation  of  Monte  Carlo 
Process  Generators 

To  develop  a process  generator  for  computer  simu- 
lation, it  is  necessary  to  find  what  is  often  called  the 
inverse  transformation  of  the  cumulative  density  function. 

In  the  case  of  the  exponential  density  function,  the  CDF 
is: 

F(t)  = l-e'Xt 

Solving  this  equation  for  t gives  the  inverse  transformation 
t = Ln (1-F ( t) ) 

Now,  if  F(t)  is  a random  fraction  uniformly  distributed 
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on  the  interval  (u,l)  then  t will  be  a random  exponential 
variate. 

The  exponential  proce&s  generator,  then  is  simply: 
t - ~ Tih  ( 1-F  { t) ) 

Or  using  the  same  symbology  as  the  simulation  program: 


ALIFI 


-1  Ln(l.-YFL) 


BETAl 

Where:  BETAl  is  the  contract  MTBF  specification 

and  YFL  is  a random  fraction. 

The  Weibull  process  generator  is  found  in  an  exactly 
parallel  fashion.  The  weibull  CDF  is: 

t ® 2 

<-«5r>  > 

F't)  * 1-e  1 

The  Weibull  location  parameter  here  will  always  be  assvr»-.:d 
to  be  zero. 

„ 82 

Ln ( 1-F (t) ) 

i/e. 


Solving  for  t:  - (r— ) 

1 


t - ei(Ln(l-F{t))) 

Since  F(t)  is  a random  fraction,  (l-F(t))  is  also  a rv»Co 
fraction,  so  that  F(t)  may  be  substituted  for  (l-F(t)}, 
and: 

l/e 

t - e^LnlFU)))  £ 

©2  is  the  shape  parameter 

9^  can  be  found  by  the  following  for  Weibull 
failures: 

E ( t)  - MTBF 

And  for  Weibull  variates,  E(t)  * e^rd/Sj  + D 


K) 


1 


/<!•/ 
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SO 

„ _ MTBF 

9i  niysj  * i) 

1/9, 

And 

‘*!^artn 

Using  the  symbology  of  the  simulation  program  for  Weibull 
failures t 


mrT„,  TBF  (-Ln(YFL)) 

ALIF1  rrrye2  ♦ ir 

Or  for  Weibull  manhour  generators: 
_ PAMH  {-Ln {YFL) ) 

AJ0B  H7e2  ♦ 1) 


l/e. 


l/e. 


The  Lognormal  generator  is  found  by  first  deriving 
a normal  process  generator,  and  then  raising  e to  the 
power  of  the  normal  variates.  Since  the  cumulative  den* 
sity  function  of  the  normal  distribution  cannot  be  solved 
for  t,  an  approximate  relationship  is  presented  here  with- 
out proof  (27:260). 

The  generator:  V - (-2Ln (YFLl) ) ^^cos  (2wYFL2) 

where  YFLl  and  YFL2  are  independent  random  fractions,  will 
produce  normal  variates  of  mean  C,  and  standard  deviation 
1.0.  To  convert  these  variates  of  mean  THU,  and  standard 
deviation  SD: 


VI  - V* (SO)  + TMU 

And  finally,  to  convert  this  to  a Lognormal  generator: 

V2  - eV1 

The  symbology  of  the  computer  simulation  is  identical  to 


that  above,  except  that: 

BMH1  - 2.718V1 
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The  Kolmogorov-Smirnov  Test 

For  Normality  of  MLSC  Distribution 
with  Mean  38.4  and  Standard  Deviation  2. a 


t__ 

's(t) 

F(t) 

|n| 

29.88 

.002 

.0015 

. f*  0 1 r' 

30.77 

.005 

.0043 

.0007 

31.68 

.012 

.0104 

.0016 

32.59 

.018 

.0244 

.0064 

33.49 

.050 

.0505 

,0005 

34.40 

.0860 

.0934 

.0074 

35.31 

.152 

. 1611 

,0091 

36.21 

.246 

.2546 

.0086 

37.12 

.362 

.3707 

.0087 

38.03 

.494 

.50 

.006 

38.94 

.618 

.6255 

.0075 

39.84 

.732 

.7389 

.0069 

40.75 

.822 

.8315 

.0095 

41.66 

.873 

.898 

.025 

42.56 

.924 

.9452 

.0212 

43.47 

.952 

.9719 

.0199 

44.38 

.971 

.9871 

. 016  l 

45.29 

.908 

.9946 

.01^6 

46.19 

.983 

.9979 

0.0 

S(t)  is  the 

empirical  function 

F(t)  is  the 

theoretical  function 

D is  the  Kolmogorov-Srairnov  statistic 

For  a = .10 

, D *=  .0836 

The  null  hypothesis  is: 

Hq:  The  distribution  is  Normal  with  parameters  38.4,  2.8 

The  alternate  hypothesis  is: 


H : The  distribution  is  not  Normal  with  parameters  38.4,  2.8 

A 

Decision  rule:  If  j D | ^ Dq  Reject  Hq 


Since  D<Dq  For  all  values  of  t,  do  not  reject 
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The  Kolmogorov -Smirnov  Teat  for 
Weibull  Distribution  of  MLSC's 
with  Parameter*  6^*43. 2 and 


t 

S(t) 

30.6 

.171 

39.33 

.703 

48.0 

.899 

56.9 

.943 

65 . 6 

.973 

74.3 

.981 

83.1 

. 988 

F(t) 

_L21 

. 318 

.147 

.539 

164 

.741 

.157 

.88 

.063 

.96 

.013 

.99 

.009 

.998 

.010 

All  remaining 
values  of  D 
are  less  than 
.01 

S(t)  is  the  empirical  function 
F(t)  is  the  theoretical  function 
D is  the  Kolmogorov-Smirov  statistic 
For  a«.10,  D *.0836 

ct 

The  null  hypothesis  is; 

Hq!  The  distribution  is  Weibull  with  parameters  43.2,  2.8 
The  alternate  hypothesis  is: 

H : The  distribution  is  not  Weibull  with  parameters  43.2,  2.8 

A 

Decision  rule:  If  |Dj  >_  Da  Reject  HQ 

Since  j D | > In  several  instances,  the  null  hypothesis 
is  rejected.  Notice  that  this  only  says  that  the  distri- 
bution is  not  Weibull  (43.2,  2.8).  No  inference  is  made 
as  to  what  the  real  distribution  is. 
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APPENDIX  E 


DECISION  CURVES 


PROBABILITY  (Should  Make 
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FLU  VARIABLES 


1.  BMC  Average  cost  per  FLU  repaired  at  base  levtl  for 

stockaqe  and  repair  of  lower  level  assemblies 
expressed  as  a fraction  of  the  FLU  unit  cost. 

This  is  the  implicit  repair  disposition  cost 
for  a 'FLU  representing  labor,  material  and 
stockage  for  lower  indenture  components  within 
the  FLU.  (e.g.  shop  reparable  units  or  modules) . 

2.  BMH  Average  manhours  to  perform  intermediate  level 

(base  shop)  maintenance  on  a removed  FLU  in- 
cluding fault  isolation,  repair  and  verification. 

3.  BRCT  Average  base  repair  cycle  time. 

4.  COND  Fraction  of  removals  expected  to  result  in 

condemnation  at  base  level. 

5.  DMC  Same  as  BMC  except  refers  to  depot  repair 

actions. 

6.  DMH  Same  as  BMH  except  refers  to  depot  level 

maintenance. 

7.  DRCT  Average  depot  repair  cycle  time. 

8.  IMH  Average  manhours  to  perform  corrective  main- 

tenance of  the  FLU  in  place  or  on  line,  in- 
cluding fault  isolation,  repair  and  verification 

9.  K Number  of  line  items  of  peculiar  support  equip- 

ment used  in  repair  of  the  FLU. 

10.  MTBF  Mean  time  between  failures  in  operating  hours 

of  the  FLU  in  the  operational  environment. 

11.  NRTS  Fraction  of  removals  expected  to  be  returned 

to  depot  for  repaid. 

12.  PA  Number  of  new  "p"  coded  reparable  assemblxe.^ 

within  the  FLU. 

13.  PAMH  Average  manhours  expended  for  preparation  and 

access  for  the  FLU.  For  example,  jacking, 
unbuttoning,  removal  of  other  units  and  hookup 
of  support  equipment. 


f 't 
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14. 

PP 

Number  of  new  "P"  coded  consumable  items  within 
the  FLU. 

15. 

QPA 

Quantity  of  like  FLU's  within  the  system. 
(Quantity  per  Application) . 

16. 

RIP 

Fraction  of  FLU  failures  which  can  be  repaired 
in  place  or  on  line. 

17. 

RMH 

Average  manhours  to  fault  isolate,  remove  and 
replace  the  FLU  and  verify  restoration  of  the 
system  to  operational  status. 

18. 

RTS 

Fraction  of  removals  expected  to  be  repaired 
at  base  level. 

19. 

SP 

Number  of  Standard  (already  stock  listed)  parts 
within  the  FLU. 

20. 

UC 

Expected  unit  cost  of  the  FLU  at  the  time  of 
initial  provisioning. 

21. 

UF 

Ratio  of  operating  hours  per  flying  hour  for 
the  FLU . 

22. 

W 

FLU  weight  in  pounds. 

PECULIAR  SUPPORT  EQUIPMENT  DATA 

1. 

CAB 

Cost  per  unit  of  support  equipment  for  base 
level . 

2. 

CAD 

Cost  per  unit  of  support  equipment  for  depot. 

3. 

DOWN 

Fraction  of  total  available  operating  time  that 

the  unit  of  support  equipment  will  be  down  for 
maintenance  and  calibration. 


SYSTEM  DATA 


1. 

BA 

Cost  of  additional  common  shop 
for  the  system. 

support  equipment 

2. 

BAA 

Available  work  time  in  the  base 
per  month. 

shop  in  manhours 

3. 

BLR 

Base  labor  rate. 

7 7' 
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BMR 

Bane  consumable  material  consumption  rate. 

5. 

DA 

Cost  of  additional  connon  depot  support  equip* 
raent  for  the  system. 

6. 

DAA 

Available  work  time  at  the  depot  in  manhours 
per  month. 

7. 

DLR 

Depot  labor  rate. 

8. 

DMR 

Depot  consumable  material  consumption  rate. 

9. 

FLA 

COBt  of  additional  common  or  peculiar  flight 
line  support. 

10. 

N 

Number  of  different  FLUs  within  the  system. 

11. 

PPHM 

Average  manhours  to  perform  a preflight  and  a 
post- flight  inspection  on  back-to-back  sorties. 

12. 

SLLRMH 

Average  manhours  to  perform  Service,  Load, 
Launch,  and  Recovery  for  one  sortie  including 
fueling,  lubrication,  munitions  loading,  etc. 

13. 

SMH 

Average  manhours  to  perform  a scheduled  periodic 
or  phased  inspection  on  the  system. 

, } 14. 

SMI 

Flying  hour  interval  between  scheduled  (periodic 
or  phased)  inspection. 

15. 

TFMH 

Average  manhours  to  perform  a through- flight 
inspection  between  back-to-back  sorties. 

PROPULSION  SYSTEM  DATA 

1. 

ARBUT 

Engine  Automatic  Resupply  and  Buildup  Time  in 
months. 

2. 

BP 

Base  engine  repair  cycle  time  in  months. 

3. 

CMRI 

Average  engine  operating  hours  between  re- 
movals. 

4. 

CONF 

Probability  of  satisfying  a demand  for  a whole 
engine. 

5. 

DP 

Depot  Engine  Repair  Cycle  Time  in  tenths. 

6. 

EPA 

Number  of  engines  per  aircraft. 
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7. 

ERMH 

Average  manhours  to  remove  and  replace  a whole 
engine  including  engine  trim  and  rurup  time. 

8. 

ERTS 

Fraction  of  engine  removals  expected  to  be  re- 
paired at  base  level.  (0  < ERTS  < 1). 

9. 

EUC 

Expected  unit  cost  of  a whole  engine. 

10. 

FC 

Fuel  cost  per  unit. 

11. 

FR 

Fuel  consumption  rate  in  gallons  per  flying 
hour.  This  rate  is  an  average  value. 

12. 

LS 

Number  of  stockage  locations  for  spare  engines. 
WEAPON  SYSTEM  DATA 

1. 

BUR 

Support  Equipment  Utilization  Rate — bas»  level. 

2. 

COB 

Annual  cost  to  operate  and  maintain  n ur>-e  of 
support  equipment  at  base  level  expressed  as 
a fraction  of  the  unit  cost  (CAB) . 

3. 

COD 

Same  as  COB  except  refers  to  depot  support 
equipment. 

4. 

DUR 

Support  Equipment  Utilization  Rate — depot  level 

5. 

EBO 

Standard  established  for  expected  backorders. 

6. 

K 

Number  of  operating  base  locations. 

7. 

MRO 

Average  manhours  per  maintenance  action  r- 
completing  on-equipment  maintenance  records. 

8. 

MRF 

Average  manhours  per  maintenance  action  for 
completing  off-equipment  maintenance  records. 

9. 

NSUB 

Number  of  systems. 

10. 

OS 

Fraction  of  total  force  deployed  tr  . v.-rscas 
locations. 

11. 

OSTCON 

Average  order  and  shipping  within  the  CONUS. 

12. 

OSTOS 

Average  order  and  shipping  time  to  overseas 
locations . 

13. 

PFFH 

Expected  peak  force  flying  hours  per  month. 

14. 

PIUP 

Operational  service  life  of  the  weapon  system 
in  years.  (Program  Inventory  Usage  Period) . 
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15. 

PHB 

Direct  productive  manhours  per  man  year  at  the 
base  level. 

16. 

PHD 

Same  as  PMB  for  depot  level. 

17. 

PSC 

Packing  and  shipping  costs  in  dollars  per 
pound  for  CONUS. 

18. 

PSO 

Same  as  PSC  for  overseas. 

19. 

RAC 

Recurring  inventory  management  cost  for  re- 
parable assembly  in  the  wholesale  system. 

20. 

SA 

Annual  base  supply  inventory  management  cost. 

21. 

SFH 

Average  sortie  length  in  flying  hours. 

22. 

SR 

Average  manhours  maintenance  action  for 
completing  supply  transaction  records. 

23. 

TD 

Cost  per  original  page  of  technical  docu- 
mentation. 

24. 

TF 

Fraction  of  total  settles  which  are  flown 
back-to-back . 

25. 

TFFH 

Expected  total  force  flying  tours  over  the 
Program  Inventory  Usage  Period. 

26. 

TR 

Average  manhours  per  maintenance  action  for 
completing  transportation  records. 

27. 

TRB 

Annual  turnover  rate  for  base  personnel. 

28. 

TRO 

Annual  turnover  rate  for  depot  personnel. 
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